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Abstract 

Additive  manufacturing  is  a  relatively  new  technique  that  is  gaining  popularity  in 
many  applications.  This  research  examines  the  possibilities  for  the  integration  of  additive 
manufacturing  (AM)  machines  in  United  States  Air  Force  civil  engineer  (CE) 
contingency  operations.  A  Delphi  study  was  conducted  that  combined  the  knowledge 
and  experience  of  experts  in  both  the  AM  industry  and  the  Air  Force  CE  community  to 
forecast  the  possible  benefits  and  drawbacks  of  this  novel  AM  application. 

The  results  of  this  Delphi  study  indicate  that  including  an  AM  machine  would  be 
beneficial  in  meeting  deployed  Air  Force  CE  requirements.  Further,  AM  technology  has 
reached  a  point  that  a  pilot  study  would  be  beneficial  to  validate  the  benefits  of  including 
an  AM  machine  in  CE  operations.  Proposed  goals  of,  and  a  design  for  this  study  are 
presented.  Further,  the  results  indicate  that  within  the  next  five  years,  AM  technology 
will  have  progressed  far  enough  that  a  full-scale  deployment  of  AM  machines  to  meet  Air 
Force  CE  contingency  requirements  will  be  beneficial. 
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A  DELPHI  STUDY  OF  ADDITIVE  MANUFACTURING  APPLICABILITY  FOR 

UNITED  STATES  AIR  FORCE  CIVIL  ENGINEER  CONTINGENCY 

OPERATIONS 

I.  Introduction 

Additive  manufacturing  is  a  relatively  new  technique  that  is  gaining  popularity  in 
many  applications.  One  of  these  developing  applications  is  the  use  of  additive 
manufacturing  machines  for  the  production  of  supplies  in  remote,  austere,  or  deployed 
locations.  United  States  Air  Force  civil  engineers  are  one  of  the  many  organizations  that 
often  labor  in  such  contingency  environments,  yet  no  research  to  date  has  addressed  the 
application  of  additive  manufacturing  for  Air  Force  civil  engineers  in  these  locations. 

This  research  endeavors  to  detennine  how  additive  manufacturing  techniques  can  be 
beneficially  applied  in  Air  Force  civil  engineer  contingency  operations  and  to  predict  the 
appropriate  timeframe  for  this  novel  application. 

General  Issue 

United  States  Air  Force  (USAF)  civil  engineers  (CEs)  are  responsible  for  the 
construction,  operation,  maintenance,  repair,  and  disposal  of  USAF  civil  infrastructure  on 
Air  Force  bases  throughout  the  United  States  and  abroad.  These  CEs  manage  a  diverse 
portfolio  of  infrastructure  that  includes  facilities,  roads,  runways,  water  distribution,  and 
other  systems.  CEs  are  responsible  for  these  systems  not  only  on  large,  primary  bases, 
but  also  in  contingency  locations  that  are  often  remote,  isolated,  and  austere. 

Maintaining  infrastructure  at  contingency  locations  poses  unique  and  significant 
challenges.  One  of  these  challenges,  and  the  focus  of  this  research,  is  the  supply  of  tools 
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and  parts  required  for  infrastructure  maintenance  activities.  Due  to  their  remote  and 
isolated  nature,  contingency  locations  often  prove  to  be  challenging  to  supply.  As  a 
result,  initial  CE  teams  typically  deploy  to  these  sites  with  a  toolkit  that  provides  them 
with  an  initial  capability  to  maintain  and  repair  infrastructure. 

These  deployable  CE  toolkits  are  known  as  Unit  Type  Code  toolkits,  or 
equipment-only  UTCs.  There  are  dozens  of  different  CE  UTCs,  each  of  which  provides  a 
unique  capability.  UTCs  can  consist  of  supplies  and  tools,  personnel,  or  both.  The  CE 
kits  examined  in  this  thesis  will  be  equipment-only  UTCs  that  contain  the  tools,  spare 
parts,  and  materials  necessary  for  construction  and  maintenance  of  infrastructure  systems. 
These  UTC  toolkits  are  designed  for  use  in  a  generalized  situation  and  are  not  tailored  to 
the  specific  environment  or  location  into  which  they  will  be  deployed.  Additionally,  they 
are  designed  to  be  air  lifted  by  small  cargo  aircraft  into  a  remote  location  and  are 
therefore  limited  in  size  and  contents.  The  ability  to  create  location-specific  tools  and 
parts  on  site,  as  needed,  could  be  beneficial  in  reducing  the  size  and  general  nature  of 
equipment-only  CE  UTCs. 

One  promising  option  for  establishing  a  more  site-specific,  compact  capability  is 
the  inclusion  of  additive  manufacturing  (AM)  machines  in  certain  CE  UTCs.  Additive 
manufacturing  is  the  process  of  constructing  objects  in  three  dimensions  by  bonding  a 
material  in  successive,  thin  layers.  The  end  result  is  a  product  that  is  built  “from  the 
ground  up”  rather  than  milled  down  from  a  block  of  material  or  cast  in  a  mold,  as  is 
common  in  conventional  manufacturing  processes.  AM  allows  more  precision  and 
flexibility  in  the  design  and  structure  of  manufactured  parts.  It  also  allows  for  the 
creation  of  thousands  of  possible  parts  or  tools  from  a  single  machine.  These  benefits  of 
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precision  and  flexibility  in  design  and  manufacturing  show  promising  possibilities  for 
addressing  the  general  nature  of  CE  UTC  toolkits,  decreasing  their  overall  size,  and 
increasing  their  capabilities. 

Problem  Statement 

Although  AM  is  a  promising  technology  for  use  in  equipment  only  UTCs,  no 
research  has  been  conducted  to  date  on  this  topic.  This  research  will  detennine  the 
qualities  of  an  AM  machine  that  would  make  it  suitable  for  such  applications  and  the 
types  of  AM  machines  that  are  currently  well  suited  for  these  UTCs.  Air  Force  CEs 
presently  do  not  know  if  current  AM  technology  has  reached  a  point  that  it  is  suitable  for 
this  application.  The  Air  Force  Civil  Engineer  Center  (AFCEC)  is  interested  in  the 
possible  AM  applications  for  CEs  and  has  deemed  this  topic  warrants  further 
investigation.  Therefore,  AFCEC  sponsored  this  research  in  an  effort  to  further 
understand  the  possible  applications  of  AM  in  CE  UTCs. 

Research  Objective  and  Investigative  Questions 

The  overall  objective  of  this  research  is  to  detennine  if  (1)  additive  manufacturing 
machines  would  be  beneficial  if  included  in  CE  equipment  UTCs  and  (2)  to  predict  the 
appropriate  timeframe  for  this  inclusion.  To  further  define  this  objective,  three  more- 
specific  investigative  questions  were  created  to  guide  this  research.  These  questions  are: 

1)  What  categories  of  AM  machine  are  currently  well  suited  for  utilization  in  CE 
equipment  UTCs? 

Many  types,  makes,  and  models  of  AM  machines  are  on  the  market  today. 
This  question  seeks  to  understand  which  of  these  various  machines  would  be 
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suitable  for  CE  applications.  This  question  does  not  look  at  companies  or 
brands,  but  instead  analyzes  the  various  raw  materials  and  build  processes 
currently  available  in  the  AM  industry. 

2)  What  attributes  make  an  AM  machine  well  suited  for  use  in  a  CE  equipment 
UTC? 

This  question  focuses  on  the  specific  attributes  necessary  in  an  AM  machine 
for  CE  contingency  applications.  It  seeks  to  understand  the  desired  qualities 
of  an  ideal  AM  machine  and  will  focus  on  machine  “-ilities”  such  as 
reliability,  usability,  quality,  maintainability,  and  others.  These  properties  are 
not  necessarily  fundamental  requirements  of  an  AM  machine,  but  knowing 
which  of  these  attributes  are  most  important  can  assist  in  selecting  the  best 
machine  for  contingency  engineering  (de  Week,  Roos,  &  Magee,  201 1:66). 

3)  Has  the  AM  industry  currently  reached  a  point  at  which  the  selected 
categories  of  AM  machines  embody  these  beneficial  attributes? 

This  question  seeks  to  understand  the  status  of  current  AM  practices  and 
future  possibilities.  AM  is  not  a  new  technology;  in  fact,  similar  methods  for 
creating  objects  layer  by  layer  have  been  in  use  since  the  1890s  (Bourell, 
Beaman  Jr.,  Leu,  &  Rosen,  2009:5).  Since  that  time,  AM  technology  has  been 
continually  progressing.  This  question  seeks  to  detennine  if  AM  technology 
has  progressed  far  enough  today,  or  if  the  technology  needs  to  further  mature, 
to  be  suitable  for  CE  contingency  applications. 
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Together,  these  questions  further  define  the  goal  of  this  thesis  effort.  They  shape 
the  literature  review,  the  methodology,  and  the  analysis.  These  questions  will  be 
referenced  later  in  this  document  when  the  methodology  and  results  are  discussed. 

Research  Approach 

This  thesis  is  the  culmination  of  multiple  stages  of  research.  The  research  begins 
with  a  literature  review  that  elucidates  the  current  state  of  AM  technologies.  This  review 
describes  the  current  AM  machine  types,  benefits,  and  limitations.  Additionally,  this 
review  describes  past  and  current  applications  of  AM  machines  in  contingency  locations. 

This  document  next  presents  the  Delphi  technique  used  in  this  research.  The 
Delphi  study  elicited  and  consolidated  the  views  of  a  panel  of  AM  and  CE  UTC  experts. 
The  objective  of  this  study  is  to  better  understand  the  current  possibilities  for  AM 
technology  in  CE  UTCs  and  to  detennine  possible  future  applications  of  this  technology. 
This  study  combined  and  refined  the  cumulative  knowledge  of  these  experts  through 
multiple  rounds  of  questions  that  address  this  topic. 

After  each  round  of  questioning,  the  Delphi  method  used  statistical  analysis  to 
highlight  the  level  of  agreement  among  the  panel  participants.  The  infonnation  gathered 
from  the  Delphi  questionnaires  and  analysis  was  used  to  generate  findings  and 
recommendations.  These  findings  illustrate  the  future  possibilities  for  AM  in  Air  Force 
CE  contingency  operations.  The  recommendations  include  possible  applications  for  AM 
machines  and  suggestions  for  further  research  that  would  be  beneficial  for  this  subject. 
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Thesis  Overview 


This  document  follows  the  traditional  five-chapter  thesis  format.  Succeeding  this 
introductory  chapter,  Chapter  II  reviews  the  current  literature  on  AM  technology. 

Chapter  III  discusses  the  method  utilized  in  this  research,  the  Delphi  study  technique. 
Chapter  IV  presents  the  opinions  of  the  panel  collected  in  the  Delphi  study  and  describes 
the  analysis  of  this  information.  Finally,  Chapter  V  discusses  the  findings  and 
recommendations  of  this  research  and  provides  suggestions  for  follow-on  research. 
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II.  Literature  Review 


Chapter  Overview 

This  chapter  summarizes  the  literature  review  conducted  for  this  research,  which 
provides  background  infonnation  on  several  topics.  First,  Air  Force  CE  UTC  utilization 
and  management  is  discussed.  Second,  an  overview  of  the  current  situation  of  the  AM 
industry  is  presented.  Third,  some  of  the  geopolitical,  economic,  and  environmental 
benefits  of  AM  are  presented.  Next,  several  current  applications  for  AM  technology  in 
military  applications  and  contingency  locations  are  examined  and  discussed.  Finally,  the 
chapter  concludes  with  a  discussion  of  the  Delphi  method,  including  its  strengths  and 
proper  application. 

Unit  Type  Codes 

USAF  CEs  are  responsible  for  the  maintenance  and  construction  of  civil 
infrastructure  on  AF  installations  worldwide.  This  includes  large,  primary  installations 
but  also  includes  smaller,  remote,  isolated,  and  austere,  contingency  locations.  When 
CEs  deploy  to  these  contingency  locations,  they  typically  do  so  with  a  toolkit  to  perfonn 
construction,  maintenance,  and  repair  activities.  These  toolkits  are  known  as  Unit  Type 
Code  toolkits,  or  UTCs. 

Air  Force  UTCs. 

The  Air  Force  defines  a  UTC  as  “a  potential  capability  focused  upon 
accomplishment  of  a  specific  mission  that  the  military  service  provides”  (United  States 
Department  of  the  Air  Force,  2006:87).  Therefore,  each  UTC  is  not  just  a  toolkit:  it  is  an 
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enabler  used  to  accomplish  a  certain  mission  or  task.  A  UTC  may  include  tools, 
equipment,  and  supplies  and  it  may  also  include  AF  personnel  (United  States  Department 
of  the  Air  Force,  2006:87).  Some  UTCs  consist  only  of  equipment,  some  contain  only 
personnel,  and  some  are  a  combination  of  both  personnel  and  equipment. 

Every  UTC  is  identified  by  several  pieces  of  information,  a  unique  number,  a 
Mission  Capabilities  Statement  (MISCAP),  a  personnel  number,  and  a  material  weight. 
The  number  that  defines  each  UTC  is  a  five-digit  alphanumeric  code,  which  uniquely 
identifies  a  UTC  and  indicates  the  functional  area  responsible  for  the  UTC.  The 
MISCAP  is  a  brief  “statement  of  the  capabilities  of  the  force  identified  by  each  UTC” 
(United  States  Department  of  the  Air  Force,  2012:66).  The  personnel  number  associated 
with  each  UTC  is  known  as  the  Authorized  Personnel  (AUTH)  number.  This  number 
indicates  the  quantity  of  personnel  assigned  to  a  specific  UTC;  it  is  zero  if  no  personnel 
are  assigned  to  a  UTC.  The  weight  for  a  UTC  indicates  the  weight  of  ah  material 
contained  in  the  kit  in  total  short  tons  (ST).  This  value  is  crucial  for  determining  the 
options  for  deploying  a  UTC.  This  number  is  zero  if  the  UTC  consists  of  personnel  only 
(United  States  Department  of  the  Air  Force,  2012:66). 

UTC  Utilization. 

When  planning  for  military  or  contingency  situations,  war  planners  use  UTCs  to 
understand  and  anticipate  the  total  manpower  and  logistics  chain  required  to  support  an 
operation  (United  States  Department  of  the  Air  Force,  2012:66).  A  war  planner 
anticipating  a  military  requirement  will  turn  to  a  list  of  UTCs  to  find  a  predefined 
capability  that  will  meet  the  need.  Thus,  a  UTC  is  the  basic  building  block  utilized  to 
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meet  peacekeeping,  humanitarian  relief,  and  rotational  operation  needs  in  contingencies 
from  small  to  large  scale  (United  States  Department  of  the  Air  Force,  2006:88). 

Equipment  UTCs  are  warehoused  and  maintained  at  a  primary  base  in  the 
continental  United  States.  When  needed,  an  equipment-only  UTC  will  be  picked  up  from 
its  storage  location  and  delivered  via  air  cargo  to  the  requisite  deployed  location.  This 
system  allows  for  the  UTC  to  be  continually  maintained  and  ready  for  rapid  deployment 
at  any  time  (United  States  Department  of  the  Air  Force,  2006:88). 

Civil  Engineer  UTCs. 

The  USAF  maintains  thousands  of  UTCs  and  of  these,  96  are  specific  to  CEs 
(Grissett,  2014).  CE-specific  UTCs  are  designated  as  “4F9XX,”  where  “XX”  indicates 
the  designation  for  a  specific  UTC.  These  UTCs  meet  a  variety  of  engineering  needs  and 
each  is  specifically  tailored  to  provide  a  capability  that  may  be  needed  in  a  wide  range  of 
contingency  environments.  Two  general  engineering  kits  will  be  examined  in  further 
detail:  the  4F9ET  Engineer  Force  Equipment  Kit  and  the  4F9RY  RED  HORSE 
Equipment  Kit.  The  MISCAPS  for  these  UTCs  are  given  in  Table  1. 
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Table  1.  4F9ET  &  4F9RY  Mission  Capability  Statements 


UTC 

MISCAP 

4F9ET 

Engineer  force  equipment  set  to  support  two  4FPET  UTCs.  Supports  missions  (including 
recovery)  to  establish,  operate,  and  sustain  contingency  operating  locations,  aerial  ports,  enroute 
bases,  natural  disaster  recovery  operations  and  joint-base  support.  Provides  equipment  for 
initial  beddown  of  bare  base  and/or  forward  operating  locations.  May  be  augmented  with  one  or 
more  4F9EF  UTCs  based  on  mission  requirements. 

4F9RY 

Red  Florse  (RH)  equipment  UTC  to  support  lead  C2  element  (hub)  of  a  deployed  RFl  squadron 
responsible  for  managing  RFI  construction  projects  in  a  theatre  of  operations.  Must  be 
combined  with  a  4FPRY  UTC  to  support  RFl  beddown.  Vehicle  maintenance,  services,  design, 
and  engineering  support  surveying,  drafting,  and  material  testing  capabilities.  Requires  a 

4F9GP  UTC  for  precision  survey  requirements  using  global  positioning  system  equipment. 
Florizontal/vertical  construction  capability  is  obtained  when  combined  with  one  or  more  of  the 
following  RH  UTC  combinations.  Horizontal  construction  teams  4F9RU/4FPRU  or 
4F9RV/4FPRV  UTCs  and/or  vertical  construction  teams  4F9RS/4FPRS  or  4F9RT/4FPRT 

UTCs.  When  combined  with  a  4FPRY,  this  UTC  contains  enhanced  logistics  and 
communication  capability. 

The  4F9ET  is  a  general  engineer  force  equipment  kit  used  for  light  construction. 

It  is  an  equipment-only  UTC  and  is  designed  to  be  paired  with  two  personnel-only  UTCs. 
When  these  three  UTCs  are  deployed  together,  the  capability  to  establish,  operate,  and 
sustain  a  contingency  location  is  delivered.  This  kit  contains  basic  tools  and  equipment 
for  electricians,  structural  craftsmen,  pavements  craftsmen,  heating  ventilation  and 
cooling  (HVAC)  technicians,  and  others.  Some  of  the  items  included  are  hammers,  saws, 
tape  measures,  pliers,  rakes,  crowbars,  concrete  floats,  drills,  chisels,  screwdrivers,  levels, 
helmets,  padlocks,  ladders,  and  other  tools  needed  to  establish  and  maintain  an  air  base 
(Air  Force  Civil  Engineer  Center,  2014). 

The  4F9RY  is  a  basic  UTC  for  heavy  construction.  This  kit,  tailored  for  use  by  a 
Rapid  Engineer  Deployable  Heavy  Operational  Repair  Squadron  Engineer  (RED 
HORSE)  unit,  has  more-robust  capability  for  construction,  paving,  and  logistics.  This  kit 
is  an  equipment-only  kit  designed  to  be  used  by  the  RED  HORSE  personnel  in  the 
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4FPRY  UTC.  The  kit  includes  many  items  that  are  in  the  4F9ET  and  adds  larger  items 
like  power  distribution  panels,  latrines,  heaters,  water  purification  systems,  a  tactical 
radio  kit,  a  welding  kit,  fuel  tanks  and  pumps,  a  skid  steer  loader,  trucks  and  tents  (Air 
Force  Civil  Engineer  Center,  2014). 

Additive  Manufacturing 

The  process  of  additive  manufacturing  is  a  relatively  new  method  for  creating 
three-dimensional  (3D)  objects  from  a  supply  of  raw  material.  AM  is  the  process  of 
constructing  objects  in  three  dimensions  by  bonding  a  material  in  successive,  thin  layers. 
The  end  result  is  a  product  that  is  built  “from  the  ground  up”  rather  than  milled  down 
from  a  block  of  material  or  cast  in  a  mold,  as  is  common  in  conventional  manufacturing 
processes.  There  are  three  main  steps  in  AM:  digital  design,  production,  and  post¬ 
processing. 

The  first  step  to  AM  production  of  any  object  is  to  digitally  create  the  object  in  a 
computer  3D  modeling  software  program.  This  can  be  accomplished  by  direct  creation  in 
the  program  or  by  laser  scanning  to  create  a  digital  model  of  a  physical  object.  The  final 
digital  model  is  then  transferred  to  the  AM  machine  for  production. 

In  the  production  phase,  the  AM  machine  receives  the  digital  model  and  produces 
the  object  utilizing  the  desired  material.  This  step  is  accomplished  one  thin  layer  at  a 
time.  Each  layer  is  fused  to  the  layer  below  it,  incrementally  building  up  a  3D  part  from 
the  supply  of  raw  material.  When  production  is  complete,  the  object  can  be  removed  for 
post-processing. 
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The  final  phase  of  AM  is  post-processing,  which  includes  all  the  steps  necessary 
to  make  the  part  produced  by  AM  complete  and  useable.  Items  manufactured  in  most 
AM  machines  will  require  several  steps  of  post-processing.  Some  of  these  steps  may 
include  cleaning,  removing  construction  supports,  or  curing  the  item  produced  in  the 
machine. 

First,  a  part  created  in  an  AM  machine  will  typically  have  leftover  raw  material 
on  or  around  its  exterior,  which  must  be  removed.  For  example,  in  poweder-based  AM 
processes,  the  item  produced  must  be  removed  from  a  block  of  excess  raw  material 
powder.  In  liquid-based  AM  processes,  the  coating  of  unhardened  liquid  raw  material 
must  be  washed  off  the  exterior  of  the  item. 

Second,  several  varieties  of  AM  machines  produce  support  columns  or  structures 
while  producing  objects  that  must  be  removed  in  post-procesing.  The  use  of  these 
supports  ensures  the  structural  integrity  of  the  item  during  the  production  process.  These 
structures  allow  AM  machines  to  create  more  complex  geometries,  but  serve  no  purpose 
for  the  final  part.  Some  supports  can  be  removed  by  dissolving  them  in  a  liquid  and 
others  must  be  mechanically  separated  by  clipping,  cutting,  or  milling. 

Third,  for  some  AM  machines,  the  objects  produced  do  not  attain  their  full 
material  properties  during  production.  Parts  produced  by  these  machines  require  some 
form  of  setting  or  curing  to  achieve  their  final  strength  and  hardness.  This  curing  is 
typically  accomplished  by  ultraviolet  light  or  by  heat. 

Additive  Manufacturing  Processes. 

Many  types  of  AM  machines  are  found  in  industry  today.  Although  every 
machine  is  similar  in  that  it  creates  a  three-dimensional  item  by  combining  multiple 
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layers  of  material,  each  type  of  machine  approaches  the  task  in  a  different  way  and  uses 
different  materials.  This  research  has  adopted  the  classification  system  established  by  the 
American  Society  for  Testing  and  Materials  (ASTM)  to  sort  the  various  types  of  machine 
into  groups.  The  ASTM  has  divided  the  various  AM  machines  into  seven  categories 
based  on  the  machine’s  production  process:  binder  jetting,  directed-energy  deposition, 
material  extrusion,  material  jetting,  powder  bed  fusion,  sheet  lamination,  and  vat 
photopolymerization  (ASTM  International,  2012: 1).  Each  category  of  AM  machine  will 
be  further  defined  in  this  section  and  a  brief  synopsis  of  each  is  given  in  Table  2. 
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Table  2.  Additive  Manufacturing  Categories  (ASTM  International,  2012 


METAL/POLYMER 


Powder  Bed  Processes 

1)  Powder  Bed  Fusion 

AM  process  in  which  thermal  energy  selectively  fuses  regions  of  successive 
layers  of  powdered  raw  material 

•  Laser  Processes 

o  Selective  Laser  Melting  (SLM)  METAL 
o  Selective  Laser  Sintering  (SLS)  POLYMER 
o  Selective  Mask  Sintering  (SMS)  MET  AT, 

•  Electron  Beam  Melting  (EBM)  METAL 

2)  Binder  Jetting 

AM  process  in  which  a  liquid  bonding  agent  is  selectively  deposited  to  join 
powdered  raw  materials 

•  Powder  Bed  Binder  Jetting  (POLYMER) 

•  3DPrinting  (METAL) 

3)  Directed-Energy  Deposition 

AM  process  in  which  focused  thermal  energy  fuses  raw  materials  by  melting  as  they  are 
being  deposited 

•  Powder  Feed  (METAL) 

•  Wire  Feed  (METAL) 


POLYMER 


4)  Vat  Photopolymerization 

AM  process  in  which  a  liquid  raw  material  in  a  vat  is  selectively  cured  by  light  activated 
polymerization 

•  Stereolithography  (SLA) 

•  Flash  Curing 

•  Film  Transfer  Imaging  (FTI) 

5)  Material  Extrusion 

AM  process  in  which  raw  material  is  selectively  dispensed  through  a  nozzle  or  orifice 

•  Fused  Deposition  Modeling  (FDM) 

6)  Material  Jetting 

AM  process  in  which  droplets  of  raw  material  are  selectively  deposited 

•  Drop-on-Demand  (DoD) 

•  Multijet  Modeling 


OTHER  (typically  paper,  sometimes  metal  or  polymer) 


7)  Sheet  Lamination 

AM  process  in  which  sheets  of  raw  material  are  bonded  to  fonn  an  object. 

•  Ultrasonic  Consolidation  (UC) 

•  Adhesive  Bonding 
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Powder  Bed  Fusion. 


Selective  laser  sintering  (SLS)  and  selective  laser  melting  (SLM)  machines  spread 
a  layer  of  powdered  material,  either  metal  or  polymer,  and  then  use  a  laser  to  melt  or 
sinter  the  material  together.  Two  benefits  of  this  process  are  that  (1)  support  structures 
are  not  required  during  production  and  (2)  multiple  layers  of  items  may  be  produced  in 
one  job.  Because  of  these  benefits,  this  process  is  a  promising  option  for  traditional  mass 
production  manufacturing.  Unfortunately,  this  process  is  very  sensitive  to  changes  in 
temperature  and  humidity  so  the  build  chamber  is  generally  heat  and  atmosphere 
controlled.  A  skin-and-core  strategy  is  typically  used:  the  outside  of  each  part  in  the 
layer  is  first  traced  and  then  the  inside  of  the  part  is  solidified.  The  interior  is  often 
solidified  more  sparsely  as  less  strength  is  typically  needed  inside  a  part  (Kuhn  &  Collier, 
2014). 

The  electron  beam  melting  (EBM)  process  is  much  like  SLS/SLM  except  it  uses 
an  electron  beam  rather  than  a  laser  to  melt  or  sinter  the  raw  material.  This  process  does 
not  require  support  structures  during  production,  but  must  be  conducted  in  a  vacuum  and 
at  high  temperatures.  As  a  result,  this  method  yields  high-strength  parts  with  low  residual 
stress  (Kuhn  &  Collier,  2014). 

Powder  Bed  Binder  Jetting. 

Powder  bed  binder  jetting  also  begins  with  a  thin  layer  of  powdered  material.  A 
binder  material  is  used  to  bond  the  materials  in  the  desired  shape.  This  bonding  material 
can  remain  in  place  for  polymer  processes.  In  metal  processes,  the  bonding  material  is 
baked  out  and  the  resulting  voids  in  the  part  are  infiltrated  with  another  material.  This 
results  in  an  inhomogeneous  material,  as  the  part  is  constructed  of  one  material  and 
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infiltrated  with  another.  Powder  bed  binder  jetting  requires  no  support  structures  and 
allows  production  of  multiple  layers  of  parts  in  one  job. 

Directed  Energy  Deposition. 

In  directed  energy  deposition,  the  powdered  raw  material  is  shot  from  a  nozzle 
toward  a  beam  of  energy,  with  which  it  merges  just  above  the  build  surface  and  the 
material  melts  and  is  added  to  the  part.  This  process  is  unique  in  that  it  can  be  used  to 
coat  existing  parts  in  addition  to  creating  new  parts.  The  raw  material  can  also  be 
continually  varied  throughout  a  job,  resulting  in  graded  materials  (Kuhn  &  Collier,  2014). 

Vat  Photopolymerization  Process. 

In  vat  photopolymerization,  a  thin  layer  of  liquid  is  activated  by  ultraviolet  (UV) 
light.  The  liquid  resin  cures  (polymerizes)  locally  in  “bullet”  shapes.  This  type  of 
machine  can  produce  very  smooth  and  rounded  shapes.  In  post  processing,  support 
structures  must  be  removed  and  UV  curing  is  required  to  cure  the  excess  material 
between  the  “bullets”  to  develop  full  part  strength.  However,  the  parts  produced  using 
this  process  are  susceptible  to  aging  problems,  such  as  increased  brittleness,  due  to  light 
and  heat  sensitivity.  This  process  is  often  used  to  create  molds  for  casting  that  are  burned 
off  after  casting  is  complete.  It  is  also  common  in  manufacturing  individually  tailored 
hearing  aids  and  braces  (Kuhn  &  Collier,  2014). 

Material  Extrusion. 

In  material  extrusion,  lines  of  a  solid  raw  material  are  extruded  from  a  heated 
orifice.  This  process  resembles  creating  a  part  by  “drawing”  it  in  3D  with  a  hot  glue  gun. 
This  method  requires  support  structures,  which  are  typically  created  from  a  second 
material  that  must  be  removed  in  post-processing  (Kuhn  &  Collier,  2014).  The  most 
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common  method  of  material  extrusion  is  Fused  Deposition  Modeling  (FDM).  This 
simple  process  is  the  basis  for  most  consumer-grade  desktop  AM  machines,  commonly 
known  as  3D  printers  (Pham  &  Gault,  1998:1270). 

Material  Jetting. 

In  material  jetting,  a  print  head  places  a  heated  material  down  layer  by  layer.  This 
process  resembles  producing  a  part  by  “printing”  it  with  an  inkjet  printer.  A  separate 
support  material  is  also  produced  the  same  way.  Polyjet  or  multijet  machines  of  this  type 
are  common.  The  materials  used  in  this  process  are  typically  photosensitive  and  are 
cured  during  post  processing  with  ultraviolet  light  to  increase  strength  (Kuhn  &  Collier, 
2014). 

Sheet  Lamination. 

Sheet  lamination  is  the  process  of  combining  solid  layers  of  a  raw  material.  In 
this  process,  sheets  or  strips  of  the  material  are  utilized.  This  process  can  be 
accomplished  by  gluing  layers  of  paper  or  plastic  together,  by  melting  layers  of  plastic 
together,  or  by  joining  layers  of  metal  together  by  welding  or  with  bolts  (Kuhn  &  Collier, 
2014).  It  is  one  of  the  oldest  varieties  of  AM,  dating  back  to  the  1890s  (Bourell,  Beaman 
Jr.,  Leu,  &  Rosen,  2009:5). 

Current  State  of  the  Additive  Manufacturing  Industry. 

AM  has  been  dubbed  a  “disruptive  technology”  (Campbell  &  Ivanova,  2013:67) 
and  has  been  identified  as  a  technology  that  may  very  well  create  a  new  “Industrial 
Revolution”  (Prince,  2014:39).  Additionally,  the  technology  is  being  explored  and 
expanded  by  anned  forces  organizations  for  military  purposes.  It  is  also  beginning  to  be 
explored  as  an  option  for  contingency  and  austere  applications.  Each  of  these 
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applications  is  important  to  review  in  considering  AM  for  CE  contingency  applications 
and  will  be  further  discussed  in  this  section. 

Additive  Manufacturing  as  a  Disruptive  Technology. 

A  disruptive  technology  is  one  that  completely  changes  the  way  an  industry  or 
process  is  operated.  More  specifically,  disruptive  technologies  are  considered  “scientific 
discoveries  that  break  through  the  usual  product/technology  capabilities  and  provide  a 
basis  for  a  new  competitive  paradigm”  (Kostoff,  Boylan,  &  Simone,  2004:142).  AM 
falls  into  this  disruptive  category  for  three  reasons.  First,  AM  is  a  new  process  that 
changes  the  traditional  manufacturing  paradigm;  second,  it  is  predicted  to  have  far- 
reaching  geopolitical  and  economic  implications;  and  third,  it  is  predicted  to  have 
significant  environmental  benefits. 

Additive  Manufacturing  Paradigm  Shift. 

The  paradigm  shift  from  conventional  manufacturing  to  AM  has  already  begun. 

In  conventional  manufacturing,  the  rule  is  to  optimize  a  part  for  manufacturing,  whereas 
with  AM,  designers  can  instead  optimize  a  part  for  its  intended  function  (Winnan,  2012). 
With  traditional  manufacturing,  a  part  must  be  designed  for  a  mold  or  to  be  milled  or 
machined.  This  typically  means  there  will  be  more  upfront  capital  costs  to  design  and 
manufacture  molds,  more  scrap  material  resulting  from  milling  or  machining,  or  both. 
With  AM,  a  part  can  be  designed  the  first  time  to  the  required  specifications  without  the 
need  to  consider  molds,  milling,  or  machining  (Garrett,  2014:71). 

Geopolitical  Implications. 

In  addition  to  changing  the  traditional  design  and  manufacturing  paradigm,  AM  is 
expected  to  have  far-reaching  geopolitical  implications.  One  of  the  predicted  political 
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implications  of  AM  is  a  shift  of  power  from  inexpensive  manufacturing  hubs  to  design 
centers  (Campbell  &  Ivanova,  2013:70).  In  the  United  States,  this  may  reduce  the 
number  of  jobs  outsourced  to  countries  overseas  with  the  lowest  labor  costs  (Bourell, 

Leu,  &  Rosen,  2009:84).  Additionally,  AM  is  predicted  to  impact  global  trade  as  it 
enables  localized  production,  rather  than  traditional,  centralized  production  and  its 
requisite  supply  chain  and  product  distribution  system.  Thus,  in  the  future,  global  trade 
may  be  more  tailored  to  transporting  digital  designs  rather  than  physical  products 
(Garrett,  2014:71). 

Economic  Implications. 

Future  AM  proliferation  may  also  shift  geopolitical  trade  and  create  a  significant 
economic  impact.  One  estimate  claims  that  the  AM  industry  will  produce  over  $5  billion 
of  products  and  services  by  2020  as  shown  in  Figure  1  (Wohlers,  201 1 :55).  Another, 
more  recent  study  of  the  industry  predicts  that  the  AM  market  will  swell  to  $2 1  billion  by 
2020  (Krauskopf,  2014).  With  AM,  producing  one  item  for  which  the  design  already 
exists  is  nearly  as  inexpensive  as  producing  many.  Therefore,  AM  is  challenging  the 
long-held  concept  of  economy  of  scale.  This  fundamental  change  in  manufacturing  is 
predicted  to  have  vast  economic  impacts  (McNulty,  Amas,  &  Campbell,  2012:5). 
Additionally,  AM  is  a  significant  boon  to  the  “just-in-time”  industry  practice.  With  AM, 
not  only  can  products  be  delivered  just  in  time,  they  can  now  be  produced  as  they  are 
ordered,  even  further  reducing  warehousing  and  logistics  requirements  (Campbell  & 
Ivanova,  2013:69). 
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Figure  1.  Prediction  of  Additive  Manufacturing  Product  and 
Services  Revenue  in  Millions  of  Dollars  (Wohlers,  2011:55) 


Environmental  Benefits. 

In  addition  to  being  an  economically  advantageous  manufacturing  technique,  AM 
is  also  expected  to  be  environmentally  beneficial.  These  environmental  benefits  arise 
from  the  additive  nature  of  the  process,  possibilities  for  reduced  transportation,  and  a 
reduction  in  manufacturing  energy  requirements.  Because  AM  is  an  additive  process,  it 
is  expected  to  substantially  reduce  waste  compared  to  traditional  subtractive 
manufacturing  (McNulty,  Arnas,  &  Campbell,  2012:5).  In  fact,  AM  has  been  dubbed  a 
“nearly  zero  waste”  process  (Garrett,  2014:72).  Additionally,  AM  may  increase  the  “near 
sourcing”  of  goods  by  providing  inexpensive  manufacturing  options  at  the  point  of  use 
for  items  that  are  traditionally  manufactured  elsewhere  and  shipped  to  the  point  of  use. 
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Through  near  sourcing,  AM  is  expected  to  reduce  the  amounts  of  global  shipping  and  air 
cargo  requirements  for  manufactured  goods  (Manners-Bell,  2012:3).  Finally,  AM  has 
been  predicted  to  reduce  the  overall  requirements  for  energy  use  in  manufacturing 
although  “energy  savings  are  product-specific  and  vary  extensively”  (McNulty,  Amas,  & 
Campbell,  2012:6).  Nevertheless,  it  is  predicted  that  AM  will  contribute  to  an  overall 
reduction  in  manufacturing  energy  use  and  carbon  emissions  (Garrett,  2014:72).  Because 
of  the  predicted  environmental  benefits  of  reduced  waste,  diminished  transportation  costs, 
and  decreased  manufacturing  energy  use,  the  AM  industry  is  considered  to  be  truly 
“environmentally- friendly”  (Campbell,  Williams,  Ivanova,  &  Garrett,  2011 :4). 

Military  Applications  for  Additive  Manufacturing. 

Due  to  the  disruptive  nature  of  AM,  the  benefits  and  possibilities  it  presents  have 
not  been  overlooked  by  the  United  States  military  services.  Significant  research  is  being 
conducted  by  the  services  and  other  military-sponsored  organizations.  A  few  of  the 
ongoing  Army,  Marine,  Navy,  and  Air  Force  AM  research  efforts  and  applications  are 
reviewed  in  this  section. 

The  Army  began  researching  AM  in  the  1990s,  looking  at  stereolithography 
(Zimmerman  &  Allen,  2013: 13).  One  of  the  most  interesting,  recent  applications  that  the 
Anny  has  employed  is  the  mobile  Expeditionary  Labs  (Ex  Labs),  which  were  delivered  to 
the  Rapid  Equipping  Force  in  2012.  These  Ex  Labs  contain  an  AM  machine  along  with 
traditional  manufacturing  equipment  and  are  rapidly  deployable  to  forward  operating 
locations  to  provide  custom  engineering  and  prototyping  (United  States  Army  Rapid 
Equipping  Force,  2014). 


21 


The  United  States  Marine  Corps  has  also  been  actively  pursuing  AM  technology. 
A  2014  report  outlined  several  AM  applications  for  the  Marines,  including  inventory 
reduction  capabilities,  reduction  in  transportation  costs,  and  reduction  in  manufacturing 
costs  (Robert  W.  Appleton  &  Company,  Inc.,  2014:25).  These  are  the  same  benefits  that 
appeal  to  many  military  individuals  and  organizations. 

The  US  Navy  has  taken  the  lead  in  AM  research  and  has  various  projects  that 
include  AM  machines.  The  recent  “Print  the  Fleet”  workshop  the  Navy  held  at  Dam 
Neck,  Virginia,  illustrates  the  importance  the  Navy  is  placing  on  AM.  This  workshop 
was  designed  to  “introduce  3D  printing  and  additive  manufacturing  to  Sailors  and  other 
[Navy]  stakeholders”  (Stinson,  2014).  Additionally,  the  Navy  is  now  utilizing  AM 
machines  in  all  four  of  its  shipyards  for  rapid  prototyping  and  custom  part  fabrication 
(Cullom,  2014).  The  Navy  is  also  experimenting  with  AM  at  sea  and  has  installed  AM 
machines  on  the  USS  Essex  (Cullom,  2014)  and  the  USS  Enterprise  (Campbell  & 
Ivanova,  2013:74). 

Finally,  the  Air  Force  is  researching  possibilities  for  the  application  of  AM. 
Currently,  the  Air  Force  employs  AM  for  “design  iteration,  prototyping,  tooling  and 
fixtures,  and  for  some  noncritical  [aircraft]  parts”  (Mack,  2013).  Recently,  the  Air  Force 
awarded  several  multimillion-dollar  contracts  that  use  AM  for  both  research  and 
production,  including  one  contract  for  F-35  parts  (3DSystems,  2012)  and  another  for 
rocket  engine  parts  (Leopold,  2014). 

Additive  Manufacturing  in  Contingency  Locations. 

Additive  manufacturing  provides  a  highly  customizable  and  self-contained 
manufacturing  process.  As  such,  it  has  been  considered  for  application  in  remote, 
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isolated,  and  austere  contingency  environments  as  a  means  of  producing  necessary  items 
while  minimizing  warehousing  requirements.  Contingency  applications  are  currently 
being  researched  by  the  Department  of  Homeland  Security,  the  National  Defense 
University,  and  the  US  Anny  and  Navy. 

The  Department  of  Homeland  Security  is  assessing  the  possible  applications  for 
AM  machines  in  disaster  response  scenarios.  They  are  currently  evaluating  the 
possibilities  for  deploying  AM  machines  to  a  disaster  location  and  providing  a  central 
library  of  digital  3D  models  that  can  be  physically  produced  anywhere  as  needed 
(Lacaze,  Murphy,  Mottern,  Corley,  &  Chu,  2014).  Additionally,  The  Center  for 
Technology  and  National  Security  Policy  at  National  Defense  University  has  recognized 
the  potential  for  AM  application  in  contingency  environments.  As  a  result,  the  center 
recently  issued  a  challenge  to  “examine  the  uses  of  additive  manufacturing  for 
humanitarian  assistance  and  disaster  relief  operations”  (McNulty,  Amas,  &  Campbell, 
2012:11). 

Furthennore,  the  US  Army  has  recognized  the  benefits  of  using  AM  in  deployed 
locations.  As  previously  noted,  the  Anny  forward  deployed  AM  machines  in  their  Ex 
Labs  in  2012  (United  States  Army  Rapid  Equipping  Force,  2014).  Finally,  the  Navy  is 
researching  the  use  of  AM  machines  in  contingencies  on  the  open  seas.  They  are 
currently  testing  the  benefits  of  AM  machines  deployed  on  the  USS  Essex  (Cullom, 
2014)  and  the  USS  Enterprise  (Campbell  &  Ivanova,  2013:74),  as  previously  discussed. 
These  applications  show  that  testing  and  researching  AM  application  in  contingency 
environments  is  moving  forward  and  many  organizations  already  recognize  the  benefits 
that  AM  machines  can  provide  in  unique  situations. 
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Summary 

This  chapter  has  reviewed  AFCE  UTC  utilization  and  management,  current  AM 
technologies,  and  the  current  state  of  the  AM  industry.  This  infonnation  is  provided  as  a 
background  to  the  research  conducted  for  this  thesis  document.  The  following  chapter, 
Chapter  III,  will  build  upon  this  information  and  present  the  methodology  utilized  in  this 
research. 
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III.  Methodology 


This  chapter  presents  the  methods  and  procedures  used  in  this  research  to 
detennine  the  suitability  of  AM  for  use  in  AFCE  deployment  toolkits,  or  UTCs.  To  make 
this  detennination,  a  panel  of  experts  was  assembled  for  a  Delphi  Study.  This  chapter 
describes  the  Delphi  method  and  its  application  in  this  research.  The  results  of  this  study 
are  presented  in  Chapter  IV. 

The  Delphi  Method 

The  Delphi  Method  was  created  out  of  necessity  in  the  early  1950s.  The  need  for 
the  methodology  arose  from  the  RAND  Corporation’s  work  on  a  US  military  project 
(Linstone  &  Turoff,  2011).  During  this  project,  significant  amounts  of  forecasting  for 
previously  unstudied  topics  were  being  undertaken.  To  ascertain  the  most  accurate 
predictions,  the  RAND  Corporation  turned  to  leading  experts  in  the  field  in  an  effort  to 
gain  valuable  insight.  RAND  solicited  input  from  these  individuals  in  several, 
anonymous  rounds  and  consolidated  the  varied  insights  in  their  report.  This  was  the  first 
research  to  utilize  what  would  come  to  be  known  as  a  Delphi  study. 

Delphi  Strengths. 

The  Delphi  approach  was  named  after  the  Oracle  at  Delphi,  a  prominent  figure  in 
ancient  Greek  mythology  who  “was  able  to  predict  the  future  with  infallible  authority” 
(Clayton,  1997:374).  The  name  is  fitting  as  a  Delphi  study  is  often  used  to  predict  the 
future  or  to  “address  what  could/should  be”  (Miller,  2006: 1).  This  stands  in  contrast  to  a 
traditional  survey  that  is  designed  to  understand  or  represent  “what  is”  (Miller,  2006). 
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Although  the  Delphi  technique  is  not  a  statistical  method  for  creating  new  knowledge,  it 
is  a  powerful  tool  for  making  the  best  use  of  available  infonnation  (Powell,  2003:380). 
The  Delphi  technique  is  well  suited  to  determining  or  developing  possible  program 
alternatives  and  to  collecting  informed  judgments  on  a  topic  that  spans  a  range  of 
disciplines  (Delbecq,  Ven,  &  Gustafson,  1975:11). 

The  Delphi  technique  is  a  good  tool  for  use  when  planning  for  the  future  and 
looking  at  program  alternatives.  This  technique  is  specifically  designed  to  “predict  or 
forecast  future  events  and  relationships  in  order  to  make  appropriate  and  reasonable  plans 
or  changes”  (Ludwig,  1997).  This  is  often  the  case  in  emerging  industry  or  when 
applying  a  new  technique  in  a  novel  application.  In  such  a  situation,  the  Delphi  method 
excels  at  predicting  future  possibilities  as  it  provides  a  “flexible  and  adaptable  tool  to 
gather  and  analyze  the  needed  data”  (Hsu  &  Sandford,  2007:5). 

The  Delphi  technique  is  also  beneficial  for  gamering  expert  judgment  in  a 
multidisciplinary  topic.  The  technique  is  specifically  designed  to  “gather  infonnation 
from  those  who  are  immersed  and  imbedded  in  the  topic  of  interest  and  can  provide  real¬ 
time  and  real-world  knowledge”  (Hsu  &  Sandford,  2007:5).  Again,  this  is  particularly 
useful  in  emerging  technologies  and  their  novel  application.  Because  the  Delphi  method 
relies  upon  targeted  experts  rather  than  random  individuals,  a  Delphi  study  is  designed  to 
combine  the  knowledge  and  opinions  of  the  participants  and  to  structure  and  organize 
their  communications  (Keeney,  Hasson,  &  McKenna,  2006:206)  in  an  area  of  uncertainty 
or  where  empirical  evidence  is  lacking  or  yet  to  be  created  (Powell,  2003:376-377). 

The  Delphi  technique  is  a  unique  tool,  suited  to  unique  research  applications.  In 
particular,  it  is  a  powerful  method  for  forecasting  future  alternatives  and  possibilities  and 
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for  gathering  cutting  edge,  real-time  and  real-world  expert  opinions.  Although  this 
method  differs  from  more  traditional  survey  or  statistical  based  methodologies,  it  is  a 
powerful  tool  when  appropriately  applied  to  predictive  research. 

Delphi  Examples. 

In  reviewing  the  literature  for  this  thesis,  two  examples  of  Delphi  studies  that 
exhibit  the  benefits  describe  above  were  discovered.  These  examples  show  the  beneficial 
application  of  the  Delphi  technique  in  research  similar  to  this  thesis  but  in  different  fields. 
The  first  example  is  in  the  biomedical  field  and  the  second  is  in  the  mental  health  field. 

In  2012,  researchers  were  studying  the  possibility  of  integrating  traditional 
Chinese  medical  (TCM)  practices  into  Western  biomedicine.  A  Delphi  panel  was 
convened  to  develop  policy  for  this  research.  This  research  was  able  to  successfully 
combine  experts  from  two  disparate  medical  fields  and  pool  their  opinions  for  applying 
TCM  in  the  biomedical  field,  illustrating  the  power  of  the  Delphi  technique  for 
combining  expertise  in  a  multidisciplinary  topic  (Chung,  Ma,  Lau,  &  Griffiths,  2012). 

A  second  Delphi  research  study  was  conducted  in  2001  in  the  mental  health  field. 
This  research  also  utilized  a  Delphi  study,  which  consisted  of  panel  members  who  were 
in  unrelated  fields,  to  forecast  future  possibilities.  In  this  study,  mental  health  counselors 
and  computer  technologists  were  combined  into  a  Delphi  panel  to  detennine  the  benefits 
and  possible  applications  of  computer  technology  in  mental  health  counseling.  This 
study  also  successfully  demonstrated  the  ability  of  the  Delphi  study  to  bring  together  two 
disparate  groups  of  experts  to  study  a  topic  that  spaned  both  fields  (Cabaniss,  2001). 
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Delphi  Features. 

Four  main  features  characterize  the  Delphi  method:  anonymity  of  participants, 
iteration,  controlled  feedback,  and  statistical  aggregation  of  group  responses  (Skulmoski, 
Hartman,  &  Krahn,  2007).  The  first  feature,  anonymity,  encourages  participants  to  share 
candid  opinions.  This  is  important  so  the  broadest  possible  viewpoints  can  be  solicited 
and  incorporated  into  the  study.  The  promise  of  anonymity  also  encourages  respondents 
to  answer  without  fear  of  retribution. 

Second,  a  Delphi  study  is  conducted  in  multiple  rounds.  The  researcher  initially 
crafts  questions  based  on  a  thorough  review  of  the  existing  body  of  knowledge  about  the 
topic  of  interest.  A  series  of  questions  is  then  carefully  crafted  and  distributed  to  the 
Delphi  participants,  typically  via  a  questionnaire.  Once  completed,  the  participants  return 
their  answers  to  the  researcher,  who  consolidates  responses.  Aggregated  responses  are 
then  used  as  a  basis  to  draft  the  questions  for  the  next  round,  which  are  again  sent  to  the 
purposive  sample  of  experts.  This  process  is  repeated  until  an  end  state  is  achieved  in 
which  the  responses  are  stable  and  further  changes  do  not  appear  likely.  At  that  point, 
final  results  are  analyzed.  The  diagram  shown  in  Figure  2  illustrates  this  iterative 
process. 
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Figure  2.  A  Four-Round  Delphi  Method  Process. 
Adapted  from  (Skulmoski,  Hartman,  &  Krahn,  2007) 


Third,  controlled  feedback  is  provided  to  participants  in  each  round.  At  the 
beginning  of  each  round,  new  questions — along  with  pertinent  responses  from  the 
previous  rounds — are  distributed  to  the  panel  members.  This  serves  two  purposes.  First 
it  ensures  accuracy  of  the  researcher’s  consolidation,  and  second,  feedback  provides 
participants  the  opinions  of  the  group  to  stimulate  further  thought  and  consideration  in 
future  rounds. 

Finally,  a  true  Delphi  study  uses  statistical  aggregation  to  interpret  the  responses 
of  the  group.  This  analysis  enables  systematic  scrutiny  of  the  responses  received  in  the 
study  (Skulmoski,  Hartman,  &  Krahn,  2007).  Often,  this  analysis  is  performed  to 
detennine  opinions  within  the  group.  In  most  Delphi  studies,  once  statistical  stability  is 
obtained,  the  Delphi  study  can  be  concluded  and  no  further  rounds  are  required. 
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Delphi  Method  Considerations. 

Several  authors  have  previously  determined  the  aspects  of  the  Delphi  study  that 
warrant  careful  scrutiny  during  the  creation  and  application  of  such  a  study  (Landeta, 
2006;  Skulmoski,  Hartman,  &  Krahn,  2007;  Linstone  &  Turoff,  2011).  Each  of  these 
articles  highlights  several  key  considerations  when  utilizing  a  Delphi  study.  The  most 
important  of  these  considerations  are  the  criteria  for  selection  of  experts  for  the  panel  and 
panel  size,  length  and  scope  of  questions  asked,  pilot  application  or  testing,  the  number  of 
rounds  to  be  conducted,  and  methods  for  encouraging  continued  participation  by  study 
participants. 

First,  when  preparing  to  conduct  a  Delphi  study,  the  research  should  consider  the 
desired  composition  of  the  Delphi  panel.  Of  primary  concern  is  the  number  of  experts 
required  for  the  study.  Proper  application  can  be  possible  with  as  few  as  eight  members 
or  as  many  as  three  hundred  or  more  (Skulmoski,  Hartman,  &  Krahn,  2007). 

Additionally,  the  criteria  for  selection  of  members  must  be  considered.  Because  the 
Delphi  study  does  not  utilize  random  selection  of  participants,  careful  scrutiny  must  be 
applied  when  selecting  members  to  cancel  bias. 

Second,  the  researcher  must  carefully  craft  the  questions  to  be  asked  during  the 
study.  One  of  the  main  decisions  to  be  made  is  the  scope  of  the  questions.  Would  open- 
ended  questions  be  preferred  or  would  pointed,  succinct  answers  better  suit  the  needs  of 
the  research?  Typically  open  ended  questions  are  utilized  in  early  rounds  of  the  Delphi 
study  for  idea  generation  and  more  pointed  questions  are  asked  in  later  rounds  to  help 
focus  the  group’s  thinking.  The  type  of  questions  to  be  used  must  be  detennined  during 
the  early  stages  and  reassessed  before  each  phase  to  maximize  the  effectiveness  of  the 
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Delphi  study  (Skulmoski,  Hartman,  &  Krahn,  2007).  Additionally,  the  amount  of  time 
required  to  answer  each  question  should  be  considered.  This  is  often  accomplished  via  a 
pilot  study  or  pre-test  of  the  questionnaire. 

Whether  or  not  to  conduct  such  a  preliminary  check  is  the  third  point  on  which  a 
researcher  should  deliberate  before  utilizing  the  Delphi  method.  A  pilot  study  or  pre-test 
will  allow  the  researcher  to  check  the  validity  and  clarity  of  the  questions  crafted  before 
submitting  them  to  the  panel.  It  will  also  provide  insight  into  the  expected  response  time, 
which  the  researcher  can  then  manipulate  if  necessary  and  use  the  resultant  time  as  a 
suggestion  for  panel  members  when  delivering  the  questionnaire. 

This  pilot  study  also  serves  as  a  check  to  ensure  ample  care  was  taken  to  avoid 
bias.  When  drafting  the  questionnaire  for  each  round,  each  of  the  questions  were 
presented  to  other  researchers  for  review  prior  to  distribution  to  the  panel.  These 
reviewers  assessed  the  questions  for  readability,  clarity,  and  bias.  Further,  the  questions 
and  statements  in  Rounds  2  and  3  were  primarily  drawn  from  the  responses  provided 
from  panel  members  in  previous  rounds  of  the  study.  This  was  done  to  limit  input  by  the 
Delphi  facilitator  and  to  decrease  bias.  Although  it  is  impossible  to  completely  remove 
bias  in  a  Delphi  study,  these  efforts  were  considered  ample  to  remove  bias  for  this 
research. 

Finally,  it  is  of  critical  importance  to  envisage  how  to  keep  panel  members 
engaged  and  encourage  their  continued  support  of  the  study.  Long  surveys  and  lengthy 
responses  will  be  difficult  for  panel  members  to  complete,  especially  as  the  Delphi  panel 
usually  consists  of  experts  who  are  likely  very  busy.  Often  it  is  necessary  to  limit  the 
number  of  questions  and  rounds  utilized  in  a  Delphi  study  to  encourage  continued 
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participation  (Landeta,  2006).  An  additional  factor  that  has  been  found  to  increase 
participation  and  promote  enthusiasm  from  Delphi  members  is  ensuring  a  quick 
turnaround  between  rounds  (Skulmoski,  Hartman,  &  Krahn,  2007).  This  further  enables 
a  short  duration  for  the  entirety  of  the  study,  keeping  it  fresh  in  the  minds  of  participants, 
which  promotes  continued  participation. 

Delphi  Study  Application 

The  primary  method  used  in  this  research  is  a  Delphi  study.  This  study  was 
designed  to  elicit  opinions  and  predictions  from  a  panel  of  experts  who  are 
knowledgeable  about  AM  and/or  CE  UTCs.  To  conduct  the  study,  four  rounds  of 
questionnaires  were  distributed  to  the  panel  participants  via  electronic  mail.  The 
questionnaires  in  each  round  were  tailored  to  generate  panel  discussion  about  possible 
AM  applications  for  CE  UTCs.  Further,  each  round  built  upon  answers  from  the 
previous  rounds.  In  this  section,  the  panel  participants  and  the  study  itself  are  described 
in  greater  detail. 

Delphi  Study  Participants. 

This  Delphi  study  began  with  the  selection  of  panel  participants.  The  panel 
consisted  of  20  individuals.  Each  of  these  individuals  was  hand  selected  based  upon  past 
experience  and  specialized  knowledge.  Two  groups  of  ten  individuals  were  selected  for 
inclusion  in  the  panel,  ten  as  AM  experts  and  ten  as  CE  UTC  experts. 

The  first  group  of  members  selected  consisted  of  ten  AM  experts.  These  panel 
members  were  chosen  from  members  of  the  America  Makes  organization.  America 
Makes  is  an  organization  based  in  Youngstown,  Ohio,  which  aims  to  “accelerate  the 
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adoption  of  additive  manufacturing  technologies  in  the  U.S.  manufacturing  sector  and  to 
increase  domestic  manufacturing  competitiveness.”  Its  members  are  individuals  who 
“are  at  the  forefront  of  new  3D  printing  materials,  technologies  and  education”  (America 
Makes,  2014).  Delphi  participants  for  this  research  were  chosen  from  America  Makes 
members  based  on  their  experience  in  academia  or  industry.  Each  participant  selected 
was  required  to  have  a  minimum  of  5  years  of  experience  in  AM  and  a  working 
knowledge  of  various  types  of  AM  processes  and  their  respective  capabilities  and 
limitations.  Table  3  presents  the  demographics  of  these  AM  experts. 

The  second  group  of  Delphi  panel  members  was  selected  for  their  experience  in 
CE  UTC  use  and  management.  These  members  were  selected  from  members  of  the  Air 
Force  Civil  Engineer  Center  (AFCEC).  AFCEC  is  the  Air  Force  organization  responsible 
for  the  planning  and  policy  for  all  CE  UTCs.  Therefore,  the  individuals  selected  for  this 
Delphi  study  were  selected  from  the  AFCEC  personnel  who  are  responsible  for  UTC 
plans  and  policies.  Panelists  were  required  to  have  a  minimum  of  3  years  of  experience 
managing  or  creating  policy  for  CE  UTCs  and  a  working  knowledge  of  CE  UTC  contents 
and  requirements.  Table  3  also  presents  the  demographics  of  these  UTC  experts. 


Table  3.  Delphi  Panel  Demographics 


AM  Experts 

CE  UTC  Experts 

Number  of  Panelists 

10 

10 

Gender 

100%  Male 

90%  Male  /  10%  Female 

Age 

38-73 

35-45 

Years  Experience 

5-23 

10-23 

Education 

Associates  Degree  -  10% 
Bachelor  of  Science  -  0% 
Doctor  of  Philosophy  -  50% 
No  Response  -  40% 

Associates  Degree  -  30% 
Bachelor  of  Science  -  50% 
Doctor  of  Philosophy  -  0% 
No  Response  -  20% 
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Conducting  the  Delphi  Study. 

To  gather  data  for  this  research  project,  a  Delphi  study  was  conducted  with  the 
individuals  described  in  the  previous  section.  This  study  was  conducted  in  four  distinct 
rounds.  Each  of  these  rounds  is  discussed  in  this  section. 

Before  the  first  round  of  this  Delphi  study  was  conducted,  a  literature  review  was 
performed  to  detennine  the  current  state  of  the  AM  industry  and  to  understand  the  current 
methods  for  deploying  CE  UTCs.  This  infonnation  is  presented  in  Chapter  II  of  this 
thesis.  This  background  infonnation  was  used  as  a  starting  point  for  the  first  round  of  the 
Delphi  study. 

One  objective  of  a  Delphi  study  is  to  detennine  if  consensus  exists  among  panel 
members.  Therefore,  it  is  important  to  define  what  will  constitute  consensus  for  this 
study.  Although  varying  levels  of  consensus  are  used  for  Delphi  studies  (Powell, 
2003:379),  for  this  research,  consensus  will  be  detennined  in  one  of  two  ways:  either  as 
75%  or  more  of  respondents  being  in  agreement  or  as  detennined  by  the  mean  of 
statistically  aggregated  responses.  The  75%  value  is  suggested  in  the  literature  as  it 
constitutes  more  than  a  simple  majority  but  is  not  so  strict  as  to  require  100%  consensus 
(Keeney,  Hasson,  &  McKenna,  2006:210).  In  other  cases,  the  mean  value  will  be  used  to 
determine  panel  consensus  as  it  is  a  measure  of  central  tendency,  which  will  indicate 
what  the  majority  of  respondents  believe  (McClave,  Benson,  &  Sincich,  201 1:57). 
Additionally,  the  mean  score  is  appropriate  because  it  is  sensitive  to  extreme  responses  or 
responses  where  only  one  panel  member  disagrees  with  the  majority  of  the  panel.  This 
will  provide  an  indication  of  disagreement  when  compared  with  the  median  responses, 
which  are  less  sensitive  to  extreme  values  (Anderson,  Sweeney,  &  Williams,  1999:67). 
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Another  goal  of  a  Delphi  study  is  to  achieve  stability  in  panel  responses.  Stability 
indicates  that  the  panel  has  sufficiently  deliberated  on  the  topic  and  each  member  has 
come  to  an  ultimate  conclusion,  whether  perfect  consensus  is  reached  or  not.  This 
stability  will  be  evident  when  the  point  of  diminishing  returns  is  reached  such  that  no 
new  infonnation  is  attained  in  subsequent  rounds  (Keeney,  Hasson,  &  McKenna, 
2006:207). 

In  Round  1  of  this  study,  a  questionnaire  was  created,  which  consisted  of  open- 
response  and  multiple-choice  questions.  This  questionnaire  was  designed  to  gather 
opinions  from  each  of  the  panel  members  concerning  AM  machine  integration  into  CE 
UTCs.  Specifically,  the  questionnaire  was  designed  to  elicit  responses  that  would  answer 
the  investigative  questions  of  this  research.  The  investigative  questions  that  were  the 
basis  for  the  Round  1  questionnaire  are: 

1)  What  categories  of  AM  machine  are  currently  well  suited  for  utilization  in 
CE  equipment  UTCs? 

2)  What  attributes  make  an  AM  machine  well  suited  for  use  in  a  CE 
equipment  UTC? 

3)  Has  the  AM  industry  currently  reached  a  point  at  which  the  selected 
categories  of  AM  machines  embody  these  beneficial  attributes? 

Each  of  the  questions  in  Round  1  stems  from  these  three  investigative  questions.  The 
Round  1  questionnaire  is  presented  in  Appendix  A  of  this  document. 

After  the  questionnaire  was  drafted,  it  was  distributed  to  the  panel  members  via 
electronic  mail  (e-mail).  Respondents  were  initially  given  one  week  to  respond  but 
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responses  received  later  were  also  accepted  and  are  included  in  this  research.  Finally,  the 
responses  received  from  each  member  were  compiled  and  sorted  into  common  categories. 

To  determine  common  categories  of  answers  among  respondents,  qualitative 
analysis  is  used  to  code  the  responses  received.  The  purpose  of  this  analysis  is  to 
“transform  data  into  findings”  (Patton,  2002:433).  The  responses  received  were  reviewed 
and  categorized  into  one  of  several  categories  that  “symbolically  assign  a  summative, 
salient,  essence-capturing,  and/or  evocative  attribute  for  a  portion  of  language- 
based.  .  .data”  (Saldana,  2009:3).  The  total  number  of  responses  that  were  detennined  to 
fall  into  each  category  was  then  tallied.  The  responses  for  several  questions  were  also 
presented  to  other  researchers  for  validation  and  to  certify  intercoder  reliability.  This 
validation  was  performed  for  no  less  than  10%  of  the  data,  as  suggested  in  the  literature 
(Lombard,  Snyder-Duch,  &  Bracken,  2002:601). 

Round  2  of  this  Delphi  study  began  with  an  analysis  of  the  common  themes  found 
in  the  previous  round.  These  themes  were  used  as  a  basis  in  designing  the  questionnaire 
for  Round  2.  The  questions  in  this  round  aimed  to  clarify  and  validate  the  responses 
received  in  Round  1 .  The  Round  2  questionnaire  consisted  primarily  of  “Likert-based” 
questions.  Likert  questions  were  first  designed  in  1932  to  measure  the  attitudes  of 
respondents  (Likert,  1932)  and  are  now  a  “well-accepted  technique  for  attitude 
measurement”  (Klooster,  Visser,  &  Jong,  2008:513).  These  questions  simplify 
participants’  responses  to  numerical  values  that  are  more  easily  analyzed  to  determine 
consensus  among  panel  members.  The  questionnaire  developed  for  Round  2  of  this 
Delphi  study  is  presented  in  Appendix  B. 
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In  phase  two  of  Round  2,  both  the  questionnaire  and  the  results  from  the  previous 
round  were  emailed  to  panel  members.  Again,  responses  from  this  round  were  compiled, 
coded,  and  analyzed.  The  results  from  this  round  were  primarily  numerical.  For  these 
numerical  questions,  simple  statistical  metrics  (i.e.,  mean,  median,  mode,  and  percentage) 
were  calculated  and  are  ample  for  analysis  (Hsu  &  Sandford,  2007:4).  After  the  results 
were  analyzed,  Round  2  was  concluded. 

In  Round  3  of  this  Delphi  study,  a  questionnaire  was  drafted  to  detennine  a  final 
forecast.  The  responses  and  themes  from  the  first  two  rounds  of  this  study  were  reviewed 
and  ten  statements  of  the  committee  members’  combined  opinions  were  drafted.  A  Likert 
scale  was  then  created  for  each  statement  to  determine  the  final  opinions  of  the  panel 
members  for  each  topic.  The  Round  3  questionnaire  is  presented  in  Appendix  C.  The 
questionnaire  was  distributed  to  panel  members  and  responses  were  once  again 
consolidated.  Simple  statistical  measures  were  again  considered  for  each  question  (i.e., 
mean,  median,  mode,  and  percentage). 

One  final  round  of  this  Delphi  study  was  then  conducted.  The  questionnaire  for 
Round  4  consisted  of  only  three  questions  and  is  presented  in  Appendix  D.  These 
questions  were  designed  to  ensure  final  stability  of  Delphi  panel  opinions  and  to  solicit 
any  final  comments.  These  results  were  compared  with  those  from  the  previous  rounds 
and  it  was  found  that  little  new  knowledge  was  gained.  Thus,  the  “point  of  diminishing 
returns”  was  reached  and  therefore  there  was  no  reason  to  conduct  any  further  rounds  of 
this  Delphi  study  (Keeney,  Hasson,  &  McKenna,  2006:207).  Accordingly,  the  Delphi 
portion  of  this  research  was  concluded. 
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Box  Plots 


To  convey  the  information  obtained  in  the  Delphi  study,  box  plots  were  created 
for  each  of  the  applicable  responses.  Box  plots  are  an  exploratory  data  analysis  tool  for 
visually  presenting  data;  these  plots  are  designed  to  show  the  central  tendency  of  data  and 
“to  rapidly  summarize  and  interpret  tabular  data”  (Williamson,  Parker,  &  Kendrick, 
1989:916).  For  this  research,  the  box  plots  are  included  to  show  the  amount  of  agreement 
or  disagreement  among  panel  members. 

To  construct  a  box  plot,  four  descriptive  statistics  were  calculated  for  each  set  of 
Likert  values.  To  calculate  these  numbers,  the  Likert  items  were  first  sorted  numerically 
from  lowest  value  to  highest  value.  Then  the  four  requisite  statistics  were  calculated, 
namely  1)  the  median,  the  middle  value  if  there  are  an  odd  number  of  data  points  or  the 
average  of  the  two  middle  values  if  there  are  an  even  number  of  data  points;  2)  the 
interquartile  range  (IQR),  which  encompasses  the  middle  50%  of  data  points  in  the  set;  3) 
the  H- spread,  which  encompasses  all  data  points  that  fall  within  1.5  times  the  IQR;  and  4) 
outliers,  any  data  points  that  fall  outside  the  //-spread.  The  box  plot  was  then  generated 
by  graphically  constructing  a  vertical  box  representing  the  IQR  with  a  horizontal  line 
through  the  box  at  the  median  value.  The  //-spread  is  depicted  by  drawing  vertical  lines 
above  and  below  the  box,  which  stop  at  the  largest  and  smallest  data  values  inside  the  H- 
spread.  Lastly,  any  outliers  are  depicted  on  the  plot  by  placing  an  asterisk  above  or  below 
the //-spread  (Williamson,  Parker,  &  Kendrick,  1989:919) 

For  this  research,  two  separate  box  plots  were  created  for  all  questions  that  have 
Likert  based  responses.  The  first  box  plot  includes  the  data  points  generated  from  the 
responses  provided  by  UTC  experts.  The  second  box  plot  was  generated  based  on  the 
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data  from  AM  expert  responses.  These  box  plots  were  placed  side  by  side  so  the 
responses  from  these  two  diverse  groups  of  respondents  can  be  visually  compared. 

Institutional  Review  Board 

This  research  contains  a  Delphi  study  that,  by  its  nature,  involves  working  with 
human  subjects.  Therefore,  the  research  is  subject  to  the  oversight  of  the  Institutional 
Review  Board  (IRB)  as  required  by  the  Code  of  Federal  Regulations  (CFR),  Title  32,  Part 
219.  The  purpose  of  this  oversight  is  to  protect  the  individuals  involved  in  the  study  and 
their  rights.  Specifically,  the  individuals  are  to  be  protected  from  reprisal  or  from 
damage  to  their  financial  standing,  employability,  or  reputation.  Additionally,  this 
oversight  ensures  that  Personally  Identifiable  Information  (PII)  for  these  individuals  is 
protected  and  not  inadvertently  released  (32  CFR  219.101,  2014). 

At  the  beginning  of  this  research,  a  plan  for  the  study  was  presented  to  the  IRB 
for  review.  This  plan  outlined  the  method  for  protecting  the  rights  of  the  individuals  who 
participated  in  the  study  and  the  manner  in  which  their  rights  and  PII  would  be 
safeguarded.  The  IRB  reviewed  this  plan  and  made  a  determination  that  the  research  was 
exempt  from  human  experimentation  requirements  as  defined  in  32  CFR  219  paragraph 
(b)  (2)  on  3 1  July  2014.  This  determination  is  included  in  Appendix  E. 

Summary 

This  chapter  has  presented  the  methodology  utilized  in  this  research  to  provide 
insight  into  the  significant  factors  that  will  detennine  the  applicability  of  AM  machines  in 
CE  UTCs.  The  Delphi  study  conducted  for  this  research  and  the  participants  in  that  study 
were  described.  The  results  of  these  methods  are  presented  in  Chapter  IV. 
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IV.  Analysis  and  Results 


The  preceding  chapter  presented  the  research  methodology  utilized  in  this  thesis. 
The  methodology  primarily  outlined  the  process  for  conducting  a  Delphi  study  designed 
to  detennine  how  the  inclusion  of  AM  machines  would  be  beneficial  in  Air  Force  CE 
UTCs  and  to  predict  the  appropriate  timeframe  for  their  inclusion.  The  Delphi  study  as 
described  in  Chapter  III  was  conducted  between  August  and  December  of  2015.  The 
results  of  each  round  of  this  study  and  an  analysis  of  these  results  are  presented  in  this 
chapter. 

Delphi  Study  Results 

For  this  research,  a  four-round  Delphi  study  was  conducted  as  described  in 
Chapter  III  of  this  thesis.  The  questionnaires  for  each  of  these  rounds  and  the  results  of 
the  responses  received  are  presented  below.  Additionally,  an  analysis  of  the  results  is 
presented  for  each  round. 

Round  1. 

In  Round  1  of  this  Delphi  study,  eight  questions  were  posed  to  the  panel 
concerning  AM  technology  and  CE  UTCs.  Five  of  these  questions  were  multiple-choice 
and  three  were  open-response.  Appendix  A  presents  the  questions  that  were  included  in 
the  first  round  of  the  Delphi  study. 

After  the  first-round  questionnaire  was  distributed,  responses  were  received  and 
compiled  from  panel  members.  In  Round  1,16  responses  were  received  from  the  20 
questionnaires  distributed.  Ten  of  the  respondents  were  UTC  experts  and  six  were  AM 
experts  as  shown  in  Table  4.  The  responses  were  then  aggregated  to  determine  the 
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percentage  of  panel  members  who  chose  a  specific  answer  for  each  question.  The  results 
for  questions  one  through  five  were  analyzed  thus  and  are  presented  in  Table  5. 


Table  4.  Delphi  Round  1  Participation 


Questionnaires  Distributed 

20 

Percentage  of 
Responses 

Total 

16 

100 

Responses  Received 

UTC  Experts 

10 

63 

AM  Experts 

6 
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Table  5.  Delphi  Round  1  Results  Questions  1-5 


1)  What  qualities  of  an  AM  machine  would  make  it  well  suited 
for  use  in  a  CE  UTC  or  in  a  deployed  or  field  operating 
environment? 


AM  Machine  Quality 

Percent  of  16  Respondents 

Usability 

75 

Reliability 

63 

Adaptability 

63 

Flexibility 

56 

Quality 

50 

Safety 

31 

Interoperability 

13 

Resilience 

6 

Other 

0 

2) 

Which  categories  of  AM  machines  show  potential  to  endure 

many  years  into  the  future? 

AM  Machine  Category 

Percent  of  12  Respondents 

Powder  Bed  Fusion 

67 

Directed  Energy  Deposition 

58 

Material  Extrusion 

50 

Binder  Jetting 

42 

Material  Jetting 

25 

Vat  Photopolymerization 

17 

41 


3)  Do  you  think  AM  technology  has  currently  reached  a  point 
where  including  an  AM  machine  in  a  UTC  would  be 
beneficial? 


Yes  or  No 

Percent  of  16  Respondents 

Yes 

38 

No 

62 

4)  When  do  you  think  technology  will  progress  far  enough  that 
inclusion  would  be  beneficial? 


Time  Range 

Percent  of  9  Respondents 

1-5  Years 

45 

5-10  Years 

33 

10-15  Years 

11 

15+  Years 

11 

5)  Which  types  of  CE  UTCs  could  be  most  benefited  by  inclusion 
of  an  AM  machine? 


Time  Range 

Percent  of  1 5  Respondents 

Light  Construction 

67 

Heavy  Construction 

40 

Surveying 

40 

Construction  Admin 

33 

Explosive  Ordinance 

Disposal 

33 

Other 

20 

Question  one  of  this  round  asked  what  qualities  of  an  AM  machine  would  be 
desirable  for  CE  UTC  applications.  The  results  from  this  study  show  that  the  Delphi 
panel  members  consider  usability  to  be  the  most  important  quality  for  a  deployed  AM 
machine.  This  quality  was  identified  by  75%  of  the  Delphi  participants  as  being 
important.  Additional  desirable  qualities  that  were  identified  are  reliability,  flexibility, 
and  adaptability.  Although  these  qualities  did  not  meet  the  75%  consensus  criterion,  they 
were  tied  for  secondary  importance,  and  more  than  half  of  the  panel  members  citied  them 
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as  important.  Finally,  quality  and  safety  were  also  identified  by  at  least  one -third  of  the 
respondents  as  an  important  quality.  Because  of  their  popularity  among  panel  members, 
these  six  qualities  were  identified  as  candidates  for  further  discussion  in  Round  2. 

In  question  two,  the  respondents  were  asked  to  identify  categories  of  AM 
machines  that  would  endure  into  the  future.  The  categories  from  ASTM  F-42 
Classification  of  Additive  Manufacturing  Processes  (ASTM  International,  2012)  were 
used  as  a  basis  for  the  responses  to  this  question.  The  panel  members  were  divided  in 
their  opinions  on  this  question,  and  no  process  stood  out  as  being  most  enduring. 
However,  the  most  common  responses  were — in  order  of  frequency — powder  bed  fusion, 
directed  energy  deposition,  material  extrusion,  and  binder  jetting.  These  processes  were 
identified  by  five  or  more  respondents,  which  constitutes  one-third  of  the  panel  members. 
Therefore,  these  four  processes  were  identified  for  further  discussion. 

Questions  three  and  four  of  the  first  round  of  this  Delphi  study  attempted  to 
determine  the  time  frame  in  which  including  an  AM  machine  in  CE  UTCs  is  expected  to 
be  beneficial.  Of  panelists  who  responded,  88%  agree  that  including  an  AM  machine 
would  be  beneficial  within  the  next  10  years,  which  constitutes  consensus.  Further,  38% 
of  respondents  agree  that  doing  so  would  be  beneficial  today. 

Question  five  of  the  first  round  of  this  Delphi  study  was  designed  to  determine  the 
appropriate  CE  applications  for  a  deployed  AM  machine.  The  results  of  this  question 
identify  that  an  AM  machine  would  be  best  suited  for  light  construction  applications  such 
as  those  the  4F9ET  UTC  is  designed  for.  Almost  65%  of  respondents  identified  this  as 
the  appropriate  application  for  a  deployed  AM  machine.  Four  other  noteworthy 
applications  were  identified  by  over  25%  of  the  panel  members:  heavy  construction, 
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construction  administration  (such  as  project  management  and  oversight),  explosive 
ordnance  disposal,  and  surveying.  Although  consensus  was  not  reached  for  any  of  these 
applications,  these  five  categories  were  identified  as  the  most  important  and  were 
selected  for  further  panel  discussion. 

The  first  five  questions  in  Round  1  of  this  Delphi  study  were  numerically  based, 
but  questions  six  and  seven  of  the  Round  1  questionnaire  were  posed  as  open-response 
questions.  The  panel  members’  answers  for  these  questions  were  compiled  and  then 
coded  to  identify  common  themes  in  responses  as  described  in  Chapter  III  of  this 
document.  The  themes  discovered  for  these  questions  are  presented  in  Table  6  along 
with  the  percentage  of  panel  members  whose  responses  included  each  theme. 

Table  6.  Delphi  Round  1  Results  Questions  6-7 

6)  What  are  some  potential  benefits  of  including  an  AM  machine  in  a  CE  UTC? 


Potential  Benefit 

Percent  of  16  Respondents 

Part  production 

63 

Rapid/on-demand  production 

31 

Prototyping/models 

25 

Tool  production 

25 

Reduced  inventory 

19 

Local  production 

13 

Better  supply  chain  options 

13 

Inexpensive 

6 

Construction 

6 

Increased  UTC  capability 

6 

Easy  to  transport 

6 
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7)  What  are  some  challenges  that  may  arise  from  including  an  AM  machine  in  a  CE 


UTC? 


Potential  Challenge 

Percent  of  15  Respondents 

Cost  prohibitive 

53 

Currently  no  added  benefit  from  including 

47 

Concerns  about  the  characteristics  of  an  AM  machine 

40 

Slower  processing  than  conventional 

33 

Power  supply  concerns 

27 

Product  qualities  lacking 

27 

AM  machine  training  challenges 

20 

New  organizational  models 

20 

Raw  material  problems 

13 

AM  is  not  just  a  machine,  need  other  components 

13 

Software/CAD  file  availability 

13 

Production  size  limited 

13 

Security 

7 

The  responses  from  question  six  were  used  to  detennine  the  most  promising 
benefits  of  including  an  AM  machine  in  a  CE  UTC.  The  panel  members  agreed  (63%  of 
respondents)  that  the  most  important  benefit  of  including  an  AM  machine  is  the  ability  to 
create  various  necessary  parts.  Additionally,  the  panel  members  identified  that  the  ability 
for  rapid,  on-demand  production  of  these  parts  would  be  beneficial  (31%).  Other  benefits 
mentioned  multiple  times  included  prototyping  and  model  building,  tool  production, 
reduced  inventory,  and  better  supply  chain  options.  These  seven  benefits  were  selected 
for  further  discussion  although  consensus  was  not  reached  for  any  of  these  benefits 
individually  in  this  round. 

In  addition  to  identifying  the  benefits,  panel  members  were  asked  to  identify  the 
possible  challenges  that  may  arise  if  an  AM  machine  is  included  in  a  CE  UTC.  The 
biggest  concern  panel  members  noted  is  how  operator  training  will  be  accomplished  for 
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the  use  of  a  new  AM  machine.  Over  half  of  the  respondents  were  concerned  about  this 
matter.  Additional  concerns  were  identified  about  the  qualities  of  an  AM  machine,  the 
material  used  in  the  machine,  the  availability  of  the  software  to  run  the  machine,  the  cost 
of  the  machine,  and  the  power  supply  for  the  machine.  Each  of  these  concerns  was 
shared  by  over  one-quarter  of  the  Delphi  participants.  Again,  though  these  were  not 
identified  by  75%  of  the  panel,  which  would  constitute  consensus,  these  six  challenges 
were  selected  for  further  analysis. 

The  final  question  of  this  round,  question  eight,  was  an  open-response  question. 
The  question  asked  panel  members  for  any  additional  comments.  In  addition,  open- 
responses  were  solicited  for  questions  one,  two,  four,  five,  six,  and  seven  of  this  round. 
The  results  of  these  open-response  comments  were  included  in  the  interpretation  of  the 
results  for  questions  one  through  seven.  The  comments  provided  from  these  open- 
responses  were  also  compiled  and  considered  when  drafting  Round  2  of  this  Delphi  study 
and  these  responses  are  presented  in  Appendix  F. 

Round  2. 

In  Round  2  of  this  Delphi  study,  the  important,  common,  and  unique  results  from 
Round  1  were  presented  for  further  discussion.  For  this  round,  nine  questions  were 
fonnulated  and  distributed  to  the  panel.  The  questions  for  Round  2  are  presented  in 
Appendix  B  of  this  thesis. 

After  Round  2  was  distributed,  responses  were  received  and  compiled  from 
Delphi  participants.  Fifteen  responses  were  received  for  Round  2  of  this  study  of  the 
twenty  questionnaires  distributed.  Eight  responses  were  from  UTC  experts  and  the 
remaining  seven  were  from  AM  experts  as  shown  in  Table  7.  These  responses  were 
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aggregated  and  coded  as  described  in  Chapter  III  of  this  thesis.  The  results  for  questions 
one  through  eight  are  presented  in  Table  8. 


Table  7.  Delphi  Round  2  Participation 


Questionnaires  Distributed 

20 

Percentage  of  Responses 

Total 

15 

100 

Responses  Received 

UTC  Experts 

8 

53 

AM  Experts 

7 

47 

Table  8.  Delphi  Round  2  Results 

1)  After  compiling  the  possible  BENEFITS  of  including  an  AM  machine  in  a  CE  UTC, 
six  common  themes  were  discovered  among  Round  1  respondents.  Of  these  six 
possible  benefits,  which  do  you  believe  are  the  most  promising  for  the  future  of 
deployed  civil  engineer  operations? 


Most  Promising  1  2  3  4  5  6  Least  Promising 


Possible  Benefit 

Mean  Score 

Box  Plot 

AM  machines  can  be  used  to  produce 
necessary  and  specialized  tools  on  site 

2.3 

7 

6 

5 

4  - 

3 

2 

1  • 

I 

6 

UTC  Experts  AM  Experts 

AM  machines  can  be  used  to  produce 
spare  parts  when  needed 

2.6 

6 

5 

4 

3 

2 

1 

1 

UTC  Experts  AM  Experts 

47 
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2) 


After  compiling  the  possible  DRAWBACKS  of  including  an  AM  machine  in  a  CE 
UTC,  six  common  themes  were  discovered  among  Round  1  respondents.  Of  these 
six  possible  drawbacks,  which  do  you  believe  are  the  most  important  to  address 
before  considering  AM  machines  for  deployed  CE  applications? 

Most  1  2  3  4  5  6  Least  Important 

Important 


Possible  Drawback 


Mean  Score 


6 


Box  Plot 


Raw  material  concerns,  such  as  sourcing 
and  safety 


2.2 


o  -I - 1 - 

UTC  Experts  AM  Experts 


Cost-prohibitive  nature  of  AM  machines 
and  materials 


3.2 


6 

5 

4 

3 

2 

1 


0 


Careful  consideration  of  AM  machine 
characteristics  in  a  deployed  environment 


UTC  Experts  AM  Experts 
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3)  80%  of  Round  1  respondents  believe  that  an  AM  machine  would  be  useful  in  at  least 

one  variety  of  UTC;  however,  no  consensus  was  reached  for  which  type  would  be 
best.  Do  you  believe  it  would  be  better  to: 


Option 

Percent  of  1 5  Respondents 

Include  an  AM  machine  in  existing  UTCs? 

23 

Create  a  new,  AM  specific  UTC? 

47 

Consider  other  methods  for  deploying  an  AM  machine? 

30 

50 


51 


6) 

How  important  do  you  believe  it  would  be  to  include  a  3D  scanner? 

Not  at  all  important  1  2  3  4  Very  important 

Mean 

Median 

Mode 

<1.5 

4 

3.5 

3 

2.5 

2 

1.5  • 

1 

0  5 

0 

T 

UTC  Experts  AM  Experts 

3.33 

3 

3 

7)  In  your  opinion,  would  a  case  study  be  useful  for  further  detennining  possible  future 
applications  of  AM  machines  in  CE  UTCs? 


Response 

Percentage  of  1 5  Respondents 

Yes 

93 

No 

7 

8)  What  aspects  should  be  tested  and  what  should  be  the  scope  of  a  pilot  or  case  study 
if  one  was  to  be  conducted? 


Aspect  to  Test 

Percent  of  1 5  Respondents 

Test  various  AM  machines/software  in  deployed 
environment 

87 

Test  actual  part  quality 

53 

Evaluate  AM  machine  "-ilities" 

53 

Test  training  plan 

33 

Test  applicability  to  current  CE  requirements 

27 

Test  assumed  benefits  (from  question  1) 

27 

Test  various  raw  materials 

27 

Test  actual  costs 

7 

In  Round  2  of  the  Delphi  study  conducted  for  this  research,  the  panel  was  asked  to 
clarify  the  results  from  Round  1  and  respond  to  additional  questions.  In  particular, 
questions  one  and  two  of  this  round  addressed  the  benefits  and  drawbacks,  determined  in 
Round  1,  of  including  an  AM  machine  in  a  CE  UTC.  From  these  questions,  it  was 
determined  that  the  most  promising  benefits  arise  from  the  ability  to  produce  spare  parts 
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and  tools  with  an  AM  machine  in  a  deployed  environment.  Concerns  about  raw  material 
procurement,  safety,  and  availability  were  identified  as  the  most  important  to  address 
before  implementation.  These  responses  were  selected  for  further  discussion  as  they  all 
scored  a  mean  of  2.0  or  lower  indicating  they  were  considered  most  important  by  the 
panel. 

In  Round  2,  question  three  addressed  the  appropriate  method  for  delivering  an 
AM  machine  to  a  deployed  location.  Three  options  were  presented:  including  the 
machine  in  an  existing  UTC,  including  the  machine  in  a  new  UTC,  or  considering 
another  method  for  deployment.  Respondents  were  also  provided  an  open-response 
section  to  clarify  or  further  expand  upon  their  answer.  Based  upon  the  responses 
received,  it  is  apparent  that  the  panel  believes  that  a  new  UTC  would  be  the  best  option 
for  AM  machine  deployment.  This  is  clear  as  almost  50%  of  the  responses  identified  this 
option  and  it  was  the  most  commonly  selected  response. 

In  question  four  of  this  round,  the  Delphi  panel  was  asked  to  consider  if  a  metal  or 
plastic/polymer  AM  machine  would  be  beneficial  for  CE  deployed  applications.  Panel 
responses  were  varied  on  this  topic.  It  appears  that  either  type  of  machine  would  be 
beneficial,  with  metal  being  slightly  favored.  However,  the  results  obtained  are  not 
strong  enough  to  conclude  that  either  type  of  machine  would  be  better  for  this 
application. 

Questions  five  and  six  ask  about  the  importance  availability  of  locally  sourced 
material  and  of  including  a  3D  scanner  in  a  deployed  AM  machine  kit.  Panelists’ 
responses  indicate  that  these  factors  are  slightly  important  to  consider.  The  panel 
determined  it  is  slightly  important  to  include  a  3D  scanner  and  the  availability  of  locally 
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sourced  materials  is  slightly  unimportant.  These  factors  were  discovered  in  Round  1  of 
this  Delphi  study  and  selected  for  further  discussion.  However,  based  upon  responses  in 
Round  2,  these  issues  do  not  appear  to  be  influential  concerns  for  the  overall  topic  and 
therefore  were  not  further  discussed  in  the  Delphi  study. 

Questions  seven  and  eight  in  Round  2  of  this  Delphi  study  are  related  to  the 
creation  of  a  pilot  or  case  study  to  gather  further  data  concerning  the  possibilities  for  AM 
deployment  in  CE  UTCs.  The  respondents  were  almost  unanimous  (14  out  of  15)  in 
affirming  the  usefulness  of  such  a  study,  which  satisfies  the  consensus  requirements  for 
this  research.  The  panel  detennined  that  a  case  or  pilot  study  should  focus  on  testing 
various  types  of  AM  machines  and  software  in  a  deployed  environment  and  validating 
the  anticipated  results  of  this  Delphi  study  in  such  an  environment.  This  opinion  was  also 
strong  enough  to  be  construed  as  consensus  from  the  committee.  The  second-most 
important  factors  to  study  are  the  “-ilities”  of  AM  machines  and  the  quality  of  the  parts 
produced  by  them  in  a  deployed  environment.  Although  fewer  than  75%  of  the  panel 
indicated  that  these  factors  should  be  considered  when  designing  a  case  or  pilot  study, 
more  than  half  agreed  they  should. 

As  in  Round  1 ,  the  final  question  for  Round  2  was  an  open-response  question 
soliciting  additional  clarification  or  final  comments  from  the  panel  members.  The 
responses  from  this  question  were  integrated  into  the  results  and  responses  for  question 
one  through  eight.  No  analysis  was  perfonned  on  this  question  individually.  Appendix  F 
presents  the  responses  received  in  this  final  question. 
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Round  3. 


After  the  responses  were  received  and  analyzed  for  Round  2,  the  questionnaire  for 
Round  3  of  this  Delphi  study  was  created.  The  questions  for  this  round  built  upon  the 
findings  obtained  in  Rounds  1  and  2  of  this  study.  The  questions  in  this  round,  however, 
were  designed  to  detennine  the  final  opinions  of  the  panel  on  the  important  topics  and 
findings  of  this  Delphi  study.  The  questions  created  for  this  round  are  presented  in 
Appendix  C  of  this  document.  In  Round  3,  participants  were  asked  to  rate  their  response 
on  a  Likert  scale  for  the  first  10  questions.  Possible  responses  ranged  from  1  to  5,  with  1 
indicating  the  panel  member  agreed  with  the  main  point  described  in  the  question  and  5 
indicating  the  member  disagreed  with  the  point.  The  scale  was  the  same  for  each 
question  and  is  presented  in  Table  9. 


Table  9.  Delphi  Study  Round  3  Response  Scale 


Response 

Likert  Value 

AGREE  with  the  main  point 

1 

SOMEWHAT  AGREE  with  the  main  point 

2 

NEITHER  agree  nor  disagree  with  the  main  point 

3 

SOMEWHAT  DISAGREE  with  the  main  point 

4 

DISAGREE  with  the  main  point 

5 

At  the  conclusion  of  Round  3,  the  responses  received  from  the  participants  were 
compiled.  In  Round  3,  sixteen  responses  were  received  of  the  twenty  questionnaires 
distributed  (Table  10).  The  mean,  median,  and  mode  for  these  compiled  responses  were 
then  calculated  to  detennine  consensus  among  panel  members.  The  results  for  questions 
one  through  ten  were  analyzed  and  are  presented  in  Table  11. 
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Table  10.  Delphi  Round  3  Participation 


Questionnaires  Distributed 

20 

Percentage  of  Responses 

Total 

16 

100 

Responses  Received 

UTC  Experts 

10 

63 

AM  Experts 

6 

37 

Table  11.  Delphi  Study  Round  3  Results 
AM  Machine  Capabilities 
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Benefits  and  Drawbacks  of  AM  Machines 
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Pilot  Study 


V) 


Creating  a  pilot  study  or  case  study  for  AM  machine 
deployment  would  be  very  beneficial  and  technology 
has  progressed  far  enough  that  this  initial  study  can  be 
performed  today.  Within  5  years  it  is  expected  that  this 
technology  will  have  progressed  enough  that  full-scale 
deployment  of  an  AM  machine  will  be  beneficial. 


Mean 

Median 

Mode 

2.0 

1.0 

1.0 

UTC  Experts 


AM  Experts 


8) 


This  study  should  primarily  be  designed  to  test  the 
ability  of  an  AM  machine  to  meet  real  world  CE 
requirements  in  a  deployed  environment. 

Mean 

Median 

Mode 

1.5 

1.0 

1.0 

X 


AM  Experts 


9) 


This  study  should  also  be  designed  to  focus  on  the 
quality  of  the  parts  produced  in  a  deployed 
environment  and  their  suitability  for  CE  applications. 


Mean 

Median 

Mode 

1.4 

1.0 

1.0 

UTC  Experts 


AM  Experts 
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10)  This  study  should  also  consider  the  “-ilities”  of  an  AM 
machine  when  used  in  a  deployed  environment.  These 
“ilities”  include  usability,  adaptability,  reliability, 
maintainability,  etc. 


Mean 

Median 

Mode 

1.3 

1.0 

1.0 

UTC  Experts  AM  Experts 


The  questions  in  Round  3  of  this  Delphi  study  can  be  divided  into  three  categories 
for  ease  of  analysis:  1)  AM  machine  capabilities,  which  includes  questions  1  -  3;  2) 
Benefits  and  drawbacks  of  AM  use  in  Air  Force  CE  contingency  operations,  which 
includes  questions  4-6;  and  3)  pilot  study  questions,  which  includes  questions  7-10. 

AM  Machine  Capabilities. 

Questions  1-3  address  the  capabilities  of  additive  manufacturing  machines. 

Some  dissension  appeared  among  panel  members  on  questions  one  and  three,  which 
address  the  type  of  AM  machine  that  should  be  used  for  Air  Force  CE  contingency 
operations.  The  panel  members  were  not  decisive  in  selecting  either  the  type  (Question 
1)  or  material  (Question  3)  that  should  be  used  for  this  application.  Part  of  this  lack  of 
agreement  may  have  arisen  due  to  the  dual  nature  of  these  questions,  which  may  have 
created  confusion  or  biased  the  results  of  the  first  part  of  the  question  if  a  panel  member 
did  not  agree  with  the  second  part  or  vice  versa.  Regardless,  throughout  the  study  there 
was  lack  of  consensus  on  these  points  and  these  topics  have  been  suggested  for  further 
research  in  Chapter  V  of  this  research.  Conversely,  there  was  strong  consensus  among 
panel  members  that  usability  is  the  most  important  quality  to  consider  for  an  AM 
machine  in  these  operations. 
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Benefits  and  Drawbacks  of  AM  Machines. 


Benefits  and  drawbacks  of  using  AM  machines  for  Air  Force  CE  contingency 
operations  were  discussed  in  questions  4-6.  The  panel  somewhat  agreed  on  the  responses 
for  each  of  these  questions.  First,  it  was  somewhat  agreed  that  the  most  promising 
benefit  that  may  be  realized  by  deploying  an  AM  machine  is  the  ability  to  create  parts 
and  tools  on  site,  and  on  demand.  Panel  members  also  somewhat  agreed  that  the  most 
important  concerns  to  address  prior  to  deploying  this  machine  are  related  to  the  raw 
material  used  for  production.  It  was  also  somewhat  agreed  that  the  most  beneficial  way 
to  deploy  an  AM  downrange  is  via  a  new  UTC. 

Pilot  Study. 

Questions  7-10  of  Round  3  of  this  Delphi  study  were  related  to  a  pilot  study.  The 
results  of  Question  7  indicate  that  panel  members  somewhat  agreed  that  conducting  a 
pilot  study  at  the  conclusion  of  this  research  would  be  beneficial.  Further,  there  was 
strong  agreement  among  the  members  about  how  the  study  should  be  designed  as  shown 
in  Question  8-10.  These  responses  indicate  that  panel  members  agree  that  if  a  pilot 
study  is  conducted,  it  should  primarily  focus  on  the  actual  ability  of  an  AM  machine  to 
meet  CE  needs  in  a  deployed  environment.  Delphi  participants  also  agreed  that  this  study 
should  be  designed  to  test  the  “-ilities”  of  both  an  AM  machine  and  the  parts  it  produces 
in  a  deployed  environment. 

In  addition  to  the  ten  Likert  questions  asked  in  Round  3,  an  open-response 
question  was  asked  at  the  conclusion  of  this  round.  The  responses  from  this  question 
were  used  as  a  basis  for  understanding  the  responses  for  the  previous  10  questions.  No 
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analysis  was  perfonned  on  this  question  individually  and  the  responses  are  included  in 
Appendix  F. 

Round  4. 

After  the  conclusion  of  Round  3,  one  last  call  was  sent  out  to  participants  to 
solicit  any  final  comments.  The  results  of  Round  3  were  distributed  to  panelists  and  these 
individuals  were  instructed  that  these  results  would  be  the  ultimate  findings  of  the  study. 
The  Round  4  questionnaire  asked  three  questions  to  guide  discussion  and  direct 
responses.  1)  Do  you  feel  these  statements  are  an  accurate  reflection  of  the  committee's 
opinions?  2)  Do  you  feel  these  statements  include  your  inputs  and  opinions?  3)  Do  you 
have  any  final  comments?  The  Round  4  questionnaire  is  presented  in  Appendix  D. 

At  the  conclusion  of  Round  4,  the  responses  received  were  compiled  for  analysis. 
In  Round  4,  only  nine  responses  were  received  of  the  twenty  questionnaires  distributed, 
seven  from  UTC  experts  and  two  from  AM  experts  as  shown  in  Table  12.  Additionally, 
Table  13  presents  a  summary  of  the  responses  received  in  Round  4. 


Table  12.  Delphi  Round  4  Participation 


Questionnaires  Distributed 

20 

Percentage  of  Responses 

Total 

9 

100 

Responses  Received 

UTC  Experts 

7 

78 

AM  Experts 

2 

22 
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Table  13.  Delphi  Study  Round  4  Results 

1)  Do  you  feel  these  statements  are  an  accurate  reflection  of  the  committee's  opinions? 


Response 

Percentage  of  9  Respondents 

Yes 

89 

No 

11 

2)  Do  you  feel  these  statements  include  your  inputs  and  opinions? 


Response 

Percentage  of  9  Respondents 

Yes 

89 

No 

11 

Of  the  responses  received  in  Round  4,  a  majority  (89%)  agreed  that  the  responses 
accurately  reflected  both  the  committee’s  opinions  and  their  own.  Although  100% 
consensus  was  not  achieved,  the  overall  responses  are  an  aggregate  of  all  the  participants’ 
beliefs  and  perfect  agreement  is  not  expected.  Significantly,  this  also  surpasses  the  75% 
requirement  in  this  research  for  consensus. 

The  responses  received  for  the  third  question  were  varied.  Many  of  these  open- 
responses  were  simply  comments  on  the  overall  study  and  participants’  interest.  Other 
responses  highlighted  topics  already  addressed  in  this  study.  These  responses  are 
presented  in  Appendix  F.  However,  after  analysis,  it  was  determined  that  no  new 
information  was  provided  and  no  new  topics  were  presented  in  the  responses  to  this 
question.  Therefore  no  analysis  was  perfonned  on  this  question  individually  but  the 
responses  to  this  question  were  used  in  the  preparation  of  this  thesis. 

At  the  conclusion  of  Round  4,  it  was  noted  that  there  was  a  significant  decrease  in 
response  rate  from  the  previous  rounds.  Additionally,  there  were  no  new  comments 
about  the  findings  of  the  study  received  from  Round  4  responses.  Rather,  the  responses 
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echoed  those  provided  in  Round  3.  This  indicates  that  stability  had  been  reached  in  the 
panel  opinions.  Further,  it  provides  evidence  that  the  “point  of  diminishing  returns”  was 
reached  in  Round  4  of  this  Delphi  study  (Keeney,  Hasson,  &  McKenna,  2006:207).  As  a 
result,  the  Delphi  study  was  concluded. 

Summary 

This  chapter  has  outlined  the  results  of  the  Delphi  study  conducted  for  this 
research.  In  Round  1,  the  panel  was  asked  to  analyze  the  current  state  of  AM  technology 
and  CE  UTCs.  In  Round  2,  these  results  were  clarified  and  expanded.  In  Round  3,  the 
conclusions  of  the  committee  were  established  and  in  Round  4,  these  conclusions  were 
presented  to  the  committee  for  final  thoughts  and  opinions.  The  results  of  this  study 
indicate  the  opinions  of  the  committee  members  on  the  various  topics  discussed.  In 
Chapter  V  of  this  document,  these  results  will  be  used  to  answer  the  investigative 
questions  posed  in  this  thesis  and  will  be  discussed  as  they  relate  to  the  thesis  research 
objective. 
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V.  Conclusions  and  Recommendations 


The  objective  of  this  chapter  is  to  present  the  conclusions  that  were  drawn  from 
this  research  concerning  AM  applications  for  deployed  Air  Force  CE  operations  and  the 
recommendations  for  future  work  in  this  area.  This  Chapter  will  first  review  the 
investigative  questions  that  guided  the  research  and  answers  to  these  questions  based 
upon  the  results  presented  in  Chapter  IV  will  be  provided.  These  answers  will  then  be 
applied  to  the  research  objective  of  this  thesis.  Next,  the  beneficial  application  of  the 
Delphi  technique  in  this  research  will  be  discussed.  Finally,  this  chapter  will  discuss 
potential  follow-on  research  and  recommendations  for  action  drawn  from  the  results  of 
this  research. 

Investigative  Questions  Answered 

To  meet  the  objective  of  this  research,  three  investigative  questions  were 
analyzed.  The  results  of  the  Delphi  study  conducted  for  this  research  were  applied  to 
these  questions  to  reach  a  final  conclusion  on  the  overall  research  objective  of  this  thesis. 
Each  of  these  questions  and  the  relevant  results  from  the  Delphi  study  are  presented 
below. 

1)  What  categories  of  AM  machine  are  currently  well  suited  for  utilization  in  CE 
equipment  UTCs? 

Several  question  presented  throughout  the  Delphi  study  were  designed  to  address 
this  question.  In  Round  3  of  the  study,  the  respondents  did  not  agree  or  disagree  with  the 
categories  of  machine  presented.  Therefore,  there  appears  to  be  some  uncertainty  about 
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this  topic.  Further  research  is  recommended  to  determine  which  category  of  machine 
would  be  well  suited  for  this  application  as  the  panel  was  unable  to  reach  a  final  opinion 
on  this  topic. 

2)  What  attributes  make  an  AM  machine  well  suited  for  use  in  a  CE  equipment 
UTC? 

In  Rounds  1  and  2  of  this  study,  the  desired  qualities  for  an  AM  machine  used  for 
CE  UTCs  were  detennined  and  prioritized.  In  Round  3,  panelists  were  asked  if  they 
agreed  with  the  ranking  established  in  the  previous  rounds.  Responses  showed  that 
panelists  agreed  that  the  following  qualities  are  most  important:  usability,  reliability, 
adaptability,  and  flexibility.  These  are  the  qualities  this  panel  of  experts  agreed  make  an 
AM  machine  well  suited  for  use  in  CE  UTCs. 

3)  Has  the  AM  industry  currently  reached  a  point  at  which  the  selected 
categories  of  AM  machines  embody  these  beneficial  attributes? 

Throughout  this  Delphi  study,  various  timeframes  were  looked  at  to  answer  this 
question.  In  Round  1 ,  almost  40%  of  respondents  agreed  that  current  technologies  would 
be  beneficial  in  a  CE  UTC.  An  additional  25%  of  respondents  believed  that  technology 
would  be  adequate  in  the  next  five  years.  Ultimately,  in  Round  3,  participants  agreed  that 
technology  has  currently  advanced  enough  for  a  pilot  or  case  study  and  that  within  five 
years,  full  scale  deployment  of  an  AM  machine  would  be  beneficial  for  CE  UTCs. 

Conclusions  of  Research 

These  three  investigative  questions  provide  context  and  background  to  meet  the 
objective  of  this  research.  The  research  objective  for  this  thesis  is  to  detennine  if  additive 
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manufacturing  machines  would  be  beneficial  if  included  in  an  Air  Force  CE  equipment 
UTC  and  to  predict  the  appropriate  timeframe  for  this  inclusion.  Based  on  the  answers  to 
these  three  investigative  questions,  the  members  pooled  in  this  Delphi  study  believe  that: 

1)  Including  an  AM  machine  in  a  new  UTC  would  be  beneficial  in  meeting 
deployed  CE  requirements. 

2)  AM  technology  has  currently  reached  a  point  at  which  a  pilot  study  would  be 
beneficial  to  validate  the  benefits  of  including  an  AM  machine  in  a  CE 
equipment  UTC. 

3)  Within  the  next  five  years,  AM  technology  will  have  progressed  far  enough 
that  a  full-scale  deployment  of  AM  machines  in  CE  UTCs  will  be  beneficial. 

These  statements,  drawn  from  experts  who  participated  in  the  Delphi  study 
conducted  for  this  research,  satisfy  the  research  objective  of  this  thesis. 

Delphi  Application 

The  use  of  the  Delphi  technique  in  this  study  proved  to  be  beneficial  for  this 
research.  The  Delphi  method  was  selected  for  its  power  to  predict  future  possibilities,  but 
was  also  useful  in  bringing  together  diverse  expertise  and  geographically  separated  panel 
members.  The  goal  of  this  research  was  to  understand  future  possibilities  for  AM 
applications  in  Air  Force  CE  applications.  This  Delphi  study  was  particularly  useful  for 
this  task  as  The  Delphi  technique  is  a  powerful  tool  for  forecasting  future  possibilities 
(Miller,  2006: 1).  Additionally,  this  research  drew  upon  the  experiences  of  two  disparate 
groups  of  experts.  Once  again,  the  Delphi  study  is  particularly  adept  at  bringing 
dissimilar  groups  together  and  combining  and  analyzing  the  panel  discussions  (Hsu  & 
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Sandford,  2007:5).  Further,  these  experts,  although  geographically  separated  were 
brought  together  by  the  Delphi  study  and  were  able  to  generate  and  share  ideas  and 
predictions.  The  nature  of  the  Delphi  study  allowed  each  participant  to  learn  from  and 
build  on  the  ideas  presented  by  other  panel  members,  and  thus  to  increase  the  overall 
productivity  of  the  study  (Keeney,  Hasson,  &  McKenna,  2006:206).  For  these  reasons, 
the  Delphi  study  was  well  suited  for  use  in  this  application  and  a  valuable  tool  in  meeting 
the  objectives  of  this  research. 

Additionally,  several  members  of  the  panel  commented  in  open-responses  about 
the  application  of  the  Delphi  study  in  this  research.  Three  participants  mentioned  that  the 
study  was  beneficial  in  exploring  new  technologies.  Further,  positive  feedback  was 
provided  about  the  overall  study  in  six  separate  responses  in  Rounds  3  and  4.  Four 
participants  commented  about  the  usefulness  of  the  Delphi  study  in  justifying  the 
proposed  pilot  study.  This  additional  feedback  provided  by  the  panel  members  further 
substantiates  the  application  of  the  Delphi  technique  in  this  research. 

Significance  of  Research 

This  research  is  the  first  of  its  type  looking  at  AM  technology  for  Air  Force  CE 
contingency  applications.  As  such,  it  provides  a  baseline  for  further  research.  This 
research  can  be  used  by  decision  makers  to  understand  the  importance  of  AM  technology, 
to  consider  the  possible  applications  of  a  deployed  AM  machine,  and  as  a  basis  for  a  pilot 
study  for  deployed  AM  applications. 

Chapter  II  of  this  thesis  contains  a  literature  review  of  current  AM  technologies. 
This  review  included  an  assessment  of  current  military  applications  of  AM  machines. 
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This  review  provides  context  and  up-to-date  information  about  the  benefits  of  AM 
technology,  which  are  important  for  decision  makers.  This  research  is  significant  as  it 
has  shown  that  AM  technology  is  an  important,  disruptive  technology  that  decision 
makers  should  understand  the  importance  of  and  not  overlook  for  CE  UTC  applications. 

Additionally,  this  research  provides  decision  makers  with  expert  opinions  on  the 
benefits  of  deploying  an  AM  machine  in  a  CE  UTC.  It  has  been  shown  that  using  an  AM 
machine  in  contingency  operations  would  be  beneficial  for  Air  Force  CEs.  Such 
application  would  provide  a  quick  and  powerful  platfonn  for  creating  necessary  parts  on 
site  and  on  demand.  However,  the  raw  materials  used  in  additive  manufacturing  should 
be  carefully  considered  and  topics  such  as  material  sourcing  and  safety  must  be  addressed 
prior  to  AM  machine  implementation.  These  benefits  and  possible  drawbacks  are  further 
detailed  in  Chapter  IV  of  this  thesis.  This  research  is  noteworthy  as  it  is  the  initial  work 
in  identifying  these  beneficial  applications  for  Air  Force  CE  decision  makers. 

Finally,  this  research  has  identified  that  experts  in  CE  UTC  management  and  the 
AM  industry  agree  that  a  pilot  study  would  be  beneficial  for  the  AF  CE  community.  A 
pilot  study  would  provide  decision  makers  with  further  information  to  determine  how 
AM  technology  can  beneficially  meet  deployed  requirements.  This  research  is  significant 
in  that  it  identifies  the  need  for  a  pilot  study  and  provides  basic  direction  for  what  the 
study  should  examine. 

This  research  is  significant  in  providing  decision  makers  with  infonnation  to 
understand  the  importance  of  AM  technology,  to  consider  the  possible  CE  applications  of 
a  deployed  AM  machine,  and  as  a  basis  for  a  pilot  study  for  contingency  AM 
applications.  Because  AM  technology  is  still  untested  for  CE  deployed  uses,  this 
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research  is  preliminary  in  nature.  However,  it  is  significant  as  a  first  step  in 
understanding  future  possibilities  for  this  new  AM  application. 

Limitations  of  Research 

Due  to  the  nature  of  this  research,  the  applicability  of  these  findings  is  limited. 
Specifically,  this  research  is  limited  in  application  to  the  Air  Force  CE  career  field.  The 
research  focused  on  the  applicability  of  AM  machines  only  for  CE  uses  and  applications. 
Therefore,  the  application  should  be  limited  to  this  single  career  field  within  the  AF  and 
should  not  be  broadened  to  the  AF  as  a  whole,  or  to  deployed  applications  in  general. 

Additionally,  the  conclusions  of  this  research  are  limited,  as  the  results  are 
opinion  based  and  are  not  indicative  of  the  participants’  respective  populations.  The 
findings  from  this  thesis  represent  the  opinions  of  the  panel  formed  for  this  purpose. 
Although  this  is  useful  information,  it  is  not  designed  to  be  a  statistical  representation  of 
the  opinions  of  a  general  population.  Therefore,  the  generalizibility  of  findings  from  this 
study  is  limited. 

Recommendations  for  Action  and  Future  Research 

Several  actionable  items  have  arisen  as  a  result  of  this  research.  Each  of  these 
items  warrants  future  analysis  and  will  further  expand  upon  the  results  of  this  thesis. 
Future  action  should  be  taken  to  institute  a  pilot  study  as  described  in  Chapter  IV  of  this 
thesis.  Additionally,  it  would  be  beneficial  to  further  assess  which  items  in  CE  UTCs  are 
good  candidates  for  contingency  AM  production,  and  which  category  of  AM  machine 
would  be  best  suited  to  produce  these  parts  through  further  research.  Each  of  these  topics 
is  discussed  further  in  this  section. 
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Pilot  Study. 

Of  foremost  importance,  this  research  recommends  that  a  pilot  study  be  created  to 
further  analyze  the  feasibility  and  benefits  of  deploying  AM  machines  to  meet  CE 
contingency  requirements.  The  experts  who  contributed  to  the  Delphi  study  that  was 
conducted  as  a  part  of  this  research  agree  that  a  pilot  study  would  be  beneficial  in  further 
validating  the  assumed  benefits  of  deploying  an  AM  machine  to  meet  CE  requirements. 
Additionally,  panel  members  agree  that  the  study  should  be  designed  to  test  the  actual 
ability  of  an  AM  machine  to  meet  CE  needs  in  a  deployed  environment.  This  study 
should  also  test  the  “-ilities”  of  both  an  AM  machine  and  the  parts  it  produces  in  a 
deployed  environment.  This  pilot  study  should  build  upon  the  findings  of  this  research 
and  the  opinions  of  the  experts  who  contributed  to  the  Delphi  study  herein  to  further 
understand  the  future  possibilities  and  benefits  of  a  deployed  CE  AM  machine.  To  meet 
these  requirements,  a  small-scale  pilot  study  should  be  created,  which  would  consist  of 
three  phases:  procurement,  training,  and  integration.  These  three  phases  are  shown  in 
Figure  3  along  with  a  notional  timeline  for  their  implementation. 


Figure  3.  Pilot  Study  Phases  and  Notional  Timeline 
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In  phase  one  of  this  pilot  study,  procurement,  further  research  should  be 
conducted  to  determine  which  AM  machine  will  best  provide  the  benefits  described  in 
Chapter  IV  of  this  research:  namely,  the  selected  machine  should  enable  production  of 
necessary  tools  and  parts  on  site,  and  on  demand,  be  able  to  produce  parts  that  will  reduce 
inventory  requirements  at  its  deployed  location,  and  free  the  location  from  some  aspects 
of  the  traditional  supply  chain.  Once  the  detennination  is  made  for  an  AM  machine  that 
will  best  provide  these  benefits,  sourcing  options  for  the  machine  should  be  considered. 
Additionally,  in  this  stage  a  detennination  for  where  the  initial  AM  machines  will  be 
deployed  should  be  made.  This  pilot  study  should  include  the  purchase  and  deployment 
of  three  AM  machines  to  large,  contingency  bases.  A  training  plan  tailored  to  the 
selected  AM  machine  should  also  be  created  for  use  in  phase  two;  two  or  three 
individuals  from  the  CE  squadron  at  each  selected  base  should  be  identified  as  AM 
machine  trainers  and  sent  to  learn  about  the  selected  printer’s  capabilities,  operations,  and 
maintenance  during  phase  one.  These  trainers  will  be  employed  in  teaching  other  CE 
personnel  about  the  selected  printer  in  phase  two.  Finally,  the  actual  procurement  should 
occur  as  a  single  purchase  to  decrease  initial  capital  costs. 

Once  the  3D  printers  are  purchased  and  delivered,  the  selected  bases  can  begin 
training,  phase  two.  The  personnel  selected  as  trainers  will  begin  instructing  base  CE 
personnel  according  to  the  training  plan  created  in  phase  one.  This  will  begin  to  build  the 
pool  of  qualified  civil  engineer  AM  machine  operators.  The  trainers  will  also  be 
responsible  for  the  overall  operation  and  maintenance  of  the  AM  machine  once  their  base 
has  taken  delivery  of  it. 
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In  phase  three,  integration,  trained  operators  will  begin  utilizing  the  AM  machine 
and  actively  pursuing  opportunities  for  its  use  and  integration  into  their  existing  CE 
operations  at  the  contingency  base.  This  integration  will  determine  if  the  benefits 
predicted  by  this  Delphi  study  will  be  achieved  in  a  real-world  contingency  environment. 
The  operators  should  also  begin  producing  parts  to  evaluate  the  “-ilities”  of  both  the 
printed  parts  and  of  the  selected  machine  during  contingency  operations.  This  phase 
should  last  for  approximately  one  year  before  the  pilot  study  is  concluded  and  the  results 
are  used  to  validate  or  dispute  the  findings  of  this  thesis  research. 

Other  Research. 

In  addition  to  the  implementation  of  a  pilot  study  as  a  follow-on  to  this  thesis, 
further  research  should  be  conducted  to  detennine  which  parts  within  current  UTCs  are 
good  candidates  for  production  by  a  deployed  AM  machine.  Future  research  should 
determine  which  parts  and  tools  in  UTCs  could  be  easily  produced  and  by  which  category 
of  machine.  Future  research  should  also  focus  on  the  supply  chain  implications  that 
would  arise  by  producing  these  parts  and  tools  by  AM  versus  the  traditional  UTC 
delivery. 

Finally,  this  proposed  research  of  producible  parts  should  be  combined  with  the 
research  conducted  in  this  thesis  to  determine  the  most  beneficial  category  of  AM 
machine  to  deploy  for  CE  applications.  This  recommendation  will  depend  largely  on 
which  parts  and  tools  are  determined  to  be  good  candidates  for  AM  production  within 
existing  UTCs.  Proper  identification  can  then  be  combined  with  this  research  to 
determine  the  most  beneficial  category  of  AM  machine  to  deploy  in  a  CE  UTC. 
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Conclusion 


This  research  has  determined  that  1)  including  an  AM  machine  in  a  new  UTC 
would  be  beneficial  in  meeting  deployed  CE  requirements.  2)  AM  technology  has 
currently  reached  a  point  at  which  a  pilot  study  would  be  beneficial  to  validate  the 
benefits  of  including  an  AM  machine  in  a  CE  equipment  UTC.  3)  Within  the  next  five 
years,  AM  technology  will  have  progressed  far  enough  that  a  full-scale  deployment  of 
AM  machines  in  CE  UTCs  is  expected  to  be  beneficial.  This  novel  application  of  AM 
technology  is  currently  untested  but  the  panel  of  experts  assembled  for  this  research 
believe  that  including  an  AM  machine  in  a  CE  UTC  would  be  beneficial  and  the  time  to 
begin  planning  for  this  integration  is  now. 
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Appendix  A  -  Delphi  Round  1  Questionnaire 


3D  Printing  Applications  in  CE  UTC  Kits 
A  Delphi  Study 

Primary  Researcher:  Captain  Seth  Poulsen 
United  States  Air  Force,  Air  Force  Institute  of  Technology 


Questions  for  Round  One  of  the  Delphi  Study 

structions:  Please  answeranvorallofthe  following  questions.  If  the  question  is  outside  of  you  area  of 
pertise,  do  notfeelthatyou  have  to  guess,  blank  answers  are  fine.  “Open  Response'  blocks  have  been 
ovidedforseveral  questions  if  you  would  liketo  explainor  elaborate  on  your  answers.  Additional  information 
iout3D  printers  and  UTCs  is  provided  as  an  attachments  this  document  if  you  would  like  additional  context. 

1)  What  qualities  of  a  3D  printer  would  make  itwell  suited  for  use  in  a  CE  UTC  or  in  a  deployed  orfield 
operating  environment?  Please  circle  up  to  FOUR: 

Quality  Safety  Usability  Resilience 

Reliability  Flexibility  Adaptability  Interoperability 

Other: _ 

Open  Response: _ 


2)  In  our  ever  changing  society,  many  technologies  become  obsolete  quickly  or  are  surpassed  by  better 
products.  With  this  in  mind,  which  categories  of  3D  printer  show  potential  to  endure  many  years  into  the 
future?  See  attached  ASTM  F-42  Classification  of  Additive  Manufacturing  Processes  for  more  information 
on  each  category  if  desired.  Please  circle  up  to  THREE: 

Powder  Bed  Fusion  Binder  Jetting  Directed  Energy  Deposition 

Vat  Material  Extrusion  Material  Jetting 

Open  Response: _ 


3)  Do  you  think3D  printing  technology  has  currently  reached  a  point  where  including  a  3D  printer  in  a  UTC 
would  be  beneficial?  Circle  ONE: 


Yes  No 
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4)  If  you  do  NOT  think  including  a  3D  printer  in  a  UTC  would  be  beneficial  yet,  when  do  you  thinktechnology 
will  progressfar  enough  that  inclusion  would  be  beneficial?Circleonly  ONE: 

1-5years  5-10years  10-15years  15+years 

Open  Response: _ 


5)  Which  types  of  CE  UTCs  could  be  most  benefited  by  inclusion  of  a  3D  Printer?  Circle  ALL  that  apply: 
Light  Construction  Heavy  Construction  Construction  Admin 

Explosive  Ordinance  Disposal  Surveying  Other: _ 

Open  Response: _ 


6)  What  are  some  potential  benefits  of  including  a  3D  printer  in  a  CE  UTC?  Open  Response: 


7)  What  are  some  challenges  that  may  arise  from  including  a  3D  printer  in  a  CE  UTC?  Open  Response: 


8)  Do  you  have  any  additional  comments? Open  Response: 
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Appendix  B  -  Delphi  Round  2  Questionnaire 


3D  Printing  Applications  in  CE  UTC  Kits 
A  Delphi  Study 

Primary  Researcher:  Captain  Seth  Poulsen 
United  States  Air  Force,  Ar  Force  Institute  of  Technology 


Questions  for  Round  Two  of  the  Delphi  Study 

Instructions:  Please  answer  anv  or  all  of  the  following  questions  to  the  extent  that  vou  are  familiar  with  the 
issues.  If  the  question  is  outside  of  your  area  of  expertise,  do  not  feel  that  you  have  to  guess,  blank  answers  are 
fine.  “Open  Response”  blocks  have  been  provided  for  several  questions  if  you  would  like  to  explain  or  elaborate 
on  your  answers. 

1)  After  compiling  the  possible  BENEFITS  of  including  a  3D  printer  in  a  CE  UTC,  six  common  themes  were 
discovered  among  Round  1  respondents.  Of  these  six  possible  benefits,  which  do  you  believe  are  the 
most  promising  forthe  future  of  deployed  civil  engineer  operations?  Please  rank  the  following  from  1 
(most  promising)  to  6  (least  promising)  by  inserting  a  rank  next  to  each  option: 

Rank  Option 

3D  printers  allow  on-demand  and  RAPID  PRODUCTION 

3D  printers  allow  production  of  PROTOTYPES  AND  MODELS  on  site  and  in  real  time 
3D  printers  can  be  used  to  produce  necessary  and  SPECIALIZED  TOOLS  on  site 
3D  printers  can  be  used  to  produce  SPARE  PARTS  when  needed 
3D  printing  capabilities  can  enable  a  REDUCTION  OF  INVENTORY  of  parts,  tools,  etc 
3D  printers  allow  independence  from  some  aspects  of  a  traditional  SUPPLY  CHAIN 

2)  After  compiling  the  possible  DRAWBACKS  of  including  a  3D  printer  in  a  CE  UTC,  six  common  themes 
were  discovered  among  Round  1  respondents.  Of  these  six  possible  drawbacks,  which  do  you  believe 
are  the  most  important  to  address  before  considering  3D  printers  for  deployed  CE  applications?  Please 
rank  the  following  from  1  (most  important  to  address)  to  6  (least  important  to  address)  by  typing  a  rank 
next  to  each  option: 

Rank  Option 

3D  printing  RAW  MATERIAL  concerns,  such  as  sourcing  and  safety 
TRAINING  requirements  for  3D  printer  operators  and  users 
COST  prohibitive  nature  of  3D  printers  and  materials 

Careful  consideration  of  3D  PRINTER  characteristics  in  a  deployed  environment 
Availability  of  necessary  SOFTWARE  and  a  “library"  of  items  to  print 
POWER  SUPPLY  for  3D  printers 

3)  80%  of  Round  1  respondents  believe  that  a  3D  printer  would  be  useful  in  at  least  one  variety  of  UTC; 
however,  no  consensus  was  reached  for  which  type  would  be  best.  Do  you  believe  it  would  be  better  to 
(highlight  one): 

Include  a  3D  printer  in  existing  UTCs? 

Create  a  new,  3D  printing  specific  UTC? 

Consider  other  methods  for  deploying  a  3D  printer? 

OPTIONAL:  Please  explain  your  reasoning  and/or  provide  suggestions  for  an  alternative  method. 


Please  click  in  this  box  to  type. 
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4)  Multiple  respondents  mentioned  the  differences  between  plastic/polymer  3D  printers  and  metal  3D 
printers.  How  beneficial  do  you  believe  inclusion  of  each  of  these  types  of  3D  printer  would  be  in  a  CE 
UTC?  Please  highlight  ONE  number  for  each: 


Plastic/Polymer: 

Not  at  all  beneficial 

1 

2 

3 

Very  beneficial 
4 

Metal: 

Not  at  all  beneficial 

1 

2 

3 

Very  beneficial 
4 

5)  Several  respondents  mentioned  the  use  of  locally  sourced  materials  for  3D  printing.  How  important  do 
you  believe  it  would  be  to  choose  a  3D  printer  for  CE  UTC  application  that  can  use  locally  sourced 
materials?  Please  highlight  ONE  number: 

Not  at  all  important  Very  important 

12  3  4 

6)  A  3D  scanner  is  a  device  that  creates  a  digital  shape  file  by  scanning  a  3D  object.  Several  respondents 
suggested  including  a  3D  scanner  in  addition  to  a  3D  printer  in  a  CE  UTC.  How  important  do  you  believe 
it  would  be  to  include  a  3D  scanner?  Please  highlight  ONE  number: 

Not  at  all  important  Very  important 

12  3  4 


7)  A  pilot  study  or  case  study  to  test  the  usefulness  and  possible  applications  of  a  3D  printer  in  CE  UTCs 
was  suggested  in  several  Round  1  responses.  In  your  opinion,  would  such  a  study  be  useful  for  further 
determining  possible  future  applications  of  3D  printers  in  CE  UTCs?  Please  highlight  ONE: 

Yes/ No 


8)  What  aspects  should  be  tested  and  what  should  be  the  scope  of  a  pilot  or  case  study  if  one  was  to  be 
conducted?  Open  Response: 
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Appendix  C  -  Delphi  Round  3  Questionnaire 


3D  Printing  Applications  in  CE  UTC  Kits 
A  Delphi  Study 

Primary  Researcher:  Captain  Seth  Poulsen 
United  States  Air  Force,  Air  Force  Institute  of  Technology 


Questions  for  Round  Three  of  the  Delphi  Study 

Introduction:  This  round  is  designed  to  “garner  consensus  and/or  highlight  differences  in  opinions  on  the  topic 
of  discussion”  as  previously  described.  I  have  compiled  the  results  of  the  previous  two  rounds  in  an  effort  to 
highlight  the  main  points  that  have  arisen  and  been  discussed.  Each  of  these  points  reflects  the  majority  of  the 
responses  from  Rounds  One  and  Two.  These  questions  will  ask  you  if  you  agree  with  the  group  on  these  points. 

Instructions:  Please  answer  aU  of  the  following  questions.  Each  question  highlights  one  of  the  main  points  of 
the  study  and  you  are  asked  to  decide  on  a  scale  of  1  to  5  if  you: 

1  =  AGREE  with  the  main  point 

2  =  SOMEWHAT  AGREE  with  the  main  point 

3  =  NEITHER  agree  nor  disagree  with  the  main  point 

4  =  SOMEWHAT  DISAGREE  with  the  main  point 

5  =  DISAGREE  with  the  main  point 

Additionally,  an  “Open  Response”  block  has  been  provided  at  the  end  of  the  questionnaire  if  you  would  like  to 
explain  or  elaborate  on  any  of  your  answers.  Thank  you  so  much  for  your  participation! 

1 )  The  types  of  3D  printer  best  suited  for  use  in  deployed  CE  operations  are  Powder  Bed  Fusion  or  Directed 
Energy  Deposition  printers.  Two  other  promising  options  are  Material  Extrusionand  Binder  Jetting. 
Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

2)  The  quality  of  a  3D  printer  most  important  to  consider  for  deployed  CE  applications  is  printer  Usability. 
Additionally,  Reliability,  Flexibility,  and  Adaptability  are  qualities  of  secondary  importance 

Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

3)  A  3D  printer  which  prints  in  metal  is  most  likely  the  best  option  for  CE  applications  but  plastic/polymer 
printers  should  not  be  completely  ruled  out.  Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

4)  The  most  promising  benefit  of  a  deployed  3D  printer  for  the  CE  community  is  the  ability  to  create 
specialized  tools  and  parts  on  site  and  on  demand.  Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

5)  The  most  important  drawbacks  to  address  before  deploying  a  3D  printer  for  CE  operations  are  the 
source,  cost,  and  safety  of  the  raw  material  for  the  printer.  Additionally,  training  requirements,  printer 
characteristics,  and  initial  costs  are  secondary  drawbacks  which  should  be  addressed. 

Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 
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6)  In  order  to  most  effectively  deploy  a  3D  printer  for  CE  use,  a  new  UTC  should  be  created  or  another 
avenue  of  deployment  should  be  considered.  Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

7)  Creating  a  pilot  study  or  case  study  for  3D  printer  deployment  would  be  very  beneficial  and  technology 
has  progressed  far  enough  that  this  initial  study  can  be  performed  today.  Within  5  years  it  is  expected 
that  this  technology  will  have  progressed  enough  that  full  scale  deployment  of  a  3D  printer  will  be 
beneficial.  Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

The  following  questions  are  in  regards  to  creating  a  pilot  or  test  study  such  as  the  one  mentioned  in  question  7 
above  and  as  previously  discussed  in  Round  2  of  this  Delphi  study. 

8)  This  study  should  primarily  be  designed  to  test  the  ability  of  a  3D  printer  to  meet  real  world  CE 
requirements  in  a  deployed  environment.  Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

9)  This  study  should  also  be  designed  to  focus  on  the  quality  of  the  parts  printed  in  a  deployed  environment 
and  their  suitability  for  CE  applications.  Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

10)  This  study  should  also  considerthe  llities"  of  a  3D  printer  when  used  in  a  deployed  environment.  These 
“ilities"  include  usability,  adaptability,  reliability,  maintainability,  etc.  Please  circle  or  highlight  one: 

AGREE  1  2  3  4  5  DISAGREE 

OPTIONAL  -  Open  Response:  Please  use  this  space  to  explain  or  elaborate  on  any  of  your  answers  or  provide 
any  additional  comments  you  may  have. 
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Appendix  D  -  Delphi  Round  4  Questionnaire 


INSTRUCTIONS:  Please  review  the  results  in  the  attached  document.  After 
reviewing  the  information,  please  consider  the  following  questions  and  provide  any  final 
comments  or  responses  you  would  like  to  contribute  or  which  have  not  been  captured  in 
the  previous  rounds  of  this  study.  A  simple  reply  email  will  suffice  for  your  response. 

Do  you  feel  these  statements  are  an  accurate  reflection  of  the  committee's  opinions? 

Do  you  feel  these  statements  include  your  inputs  and  opinions? 

Do  you  have  any  final  comments? 
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Appendix  E  -  IRB  Exemption  Letter 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 
WRIGHT-PATTERSON  AIR  FORCE  BASE  OHIO 


31  July  2014 


MEMORANDUM  FOR  DR.  VANCE  VALENCIA 

FROM:  Jeffrey  A.  Ogden.  PhD. 

AFIT  IRB  Research  Reviewer 
2950  Hobson  Way 

Wright-Patterson  AFB.  OH  45433-7765 

SUBJECT :  Approval  for  exemption  request  from  human  experimentation  requirements  (3  2  CFR 
219,  DoDD  32 1 6.2  and  AFI 40-402)  for  3D  Printing  Delphi  Study 

1 .  Your  request  was  based  on  the  Code  of  Federal  Regulations,  title  32,  part  219,  section  101, 
paragraph  (b)  (2)  Research  activities  that  involve  the  use  of  educational  tests  (cognitive, 
diagnostic,  aptitude,  achievement),  survey  procedures,  interview  procedures,  or  observation  of 
public  behavior  unless:  (i)  Information  obtained  is  recorded  in  such  a  manner  that  human 
subjects  can  be  identified,  directly  or  through  identifiers  linked  to  the  subjects;  and  (ii)  Any 
disclosure  of  the  human  subjects 5  responses  outside  the  research  could  reasonably  place  the 
subj  eets  at  risk  of  criminal  or  civil  liability  or  be  damaging  to  the  subj  ects  ’  financial  standing, 
employability,  or  reputation. 

2.  Your  study  qualifies  for  this  exemptionbecause  you  are  not  collecting  sensitive  data,  which 
could  reasonably  damage  the  subjects’  financial  standing,  employability,  or  reputation.  Further, 
you  are  not  collecting  and  reporting  any  demographic  data  which  could  realistically  be  expected 
to  map  a  given  response  to  a  specific  subject. 

3 .  This  determination  pertains  only  to  the  Federal,  Department  of  Defense,  and  Air  Force 
regulations  that  govern  the  use  of  human  subjects  in  research.  Further,  if  a  subject’s  future 
response  reasonably  places  them  at  risk  of  criminal  or  civil  liability  or  is  damaging  to  their 
financial  standing,  employability,  or  reputation,  you  are  required  to  file  an  adverse  event  report 
with  this  office  immediately. 


7/31/2014 


X 


Jeffrey  A.  Ogden 


Jeffrey  A.  Oaden,  Ph.D. 

IRB  Exempt  Determination  Official 
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Appendix  F  -  Open-Response  Question  Submissions 


Round  1  Open-Response 

Not  practical  for  a  fire  department 

You  should  consider  adding  3D  scanning  capability  to  the  UTC  also.  Can  support  reverse 
engineering  of  parts  or  scanning  of  structure 

FDM  machines  have  been  attempted  for  use  in  space  with  limited  success  and  application.  For  a 
CE  UTC,  however,  the  applications  would  be  more  extensive  and  immediate.  It  would  be 
good  to  do  a  trial  run  and  gain  some  experience  using  a  simple  FDM  setup  to  determine 
if  it  can  solve  some  field  problems.  Later,  a  metal  system  could  be  used  for  wider 
applications  &  the  more  robust  tools  included  in  a  CE  UTC. 

The  existing  benefits  are  significant  and  near-term  realizable  improvement  will  be  dramatic  - 
the  key  will  be  thoughtful  integration  and  appropriate  exploitation. 

Potential  is  limited  to  environment  and  stage  of  build  up 

More  potential  during  sustainment  than  bare  base  arrival 

From  my  limited  research,  seems  to  be  a  great  technology  with  great  potential... 

I  can  envision  it  being  used  at  structures  shops  performing  R&D  or  even  those  that  have 
specialized  missions,  i.e.  those  that  support  missile/launch  facilities  etc.,  where  on 
occasion  they're  tasked  to  create  a  special  tool  or  part  not  found  on  a  commercial  shelf. 
Other  than  that,  I'm  not  convinced  the  technology  is  advanced  enough  to  field  as  a 
contingency/deployable  asset  thus  far 
I  honestly  feel  we  do  not  have  a  need  for  this  tool  at  this  point. 

Maybe  if  it  could  print  or  cast  heavy  metals  it  would  be  of  use 
Printer  would  require  polymer  and  metal  printing 

Main  application  would  be  for  use  with  build  of  shaped  charges  and  charge  containers 

Secondary  would  be  sustainment/repair  of  tools 

Benefits  for  application  for  training,  i.e.  printing  of  UXO  training  aids 

Round  2  Open-Response 

Again,  I  don't  believe  this  is  practical  for  a  UTC  or  deployed  environment. 

I  don't  feel  the  cost  of  the  unit  or  printing  materials  makes  purchasing  and  maintaining  a  unit 
like  this  very  worthwhile. 

Deploying  and  going  to  war  takes  us  back  to  the  basics 

This  type  of  technology  would  not  be  used  or  not  utilized 

Scanning  is  external  only,  need  CAT  scanning  ability  to  capture  internal  structure 

Should  be  included  in  a  UTC,  3D  printer  is  a  tool  like  a  hammer  or  screw  now 

Recommend  a  UTC  specific  to  3D  printing 

Cut  cost  and  save  materials  by  preventing  a  3D  printer  being  postured  in  (for  example)  each 
4F9ET  UTC  and  allow  the  option  to  be  called  up  when  needed. 

Composition  of  a  UTC  could  be  reconstituted  and  reconfigured  if  a  3DPrinter  were  included 
Tools  that  are  used  infrequently  could  be  deleted  and  replace  by  an  electronic  file 
Could  be  used  to  add  fixtures  or  features  to  existing  tools,  reducing  the  array  of  tools  that  need 
to  be  included  in  the  UTC  or  expanding  the  use  of  various  tools. 

Review  the  recent  Marine  Expeditionary  Logistics  (ExLog)  game  involving  3D  Systems  printers 
and  Geomagic  software.  Also,  review  recent  Rapid  Equipment  Force  mobile  fab  lab 
operations  in  theater. 
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As  and  HVAC  technician,  I  see  benefits  in  printing  fittings  and  generic  tools 

Is  it  economically  smarter  to  print  new  vs  buy  new? 

Investigate  which  UTCs  would  have  a  need  for  the  printer 

One  probably  needs  to  bundle  the  "3D  printer"  with  personnel  capabilities  and  part/software 
capabilities  in  application  specific  scenarios  (e.g.  in  existing  UTCs);  so  there  would  be 
different  flavors  of  "3D  printers"  for  different  scenarios 

To  make  a  hammer  using  metal  3D  printing  might  seem  like  a  good  idea  but  once  you  take  all 
the  above  things  into  consideration  that  might  not  be  the  case 

We  should  only  apply  3D  printing  where  it  makes  sense. 

I  saw  on  the  news  recently  that  NASA  deployed  a  3D  printer  to  the  international  Space 

Station. ..Think  it's  a  great  asset  for  an  operation  of  that  kind  but  still  not  convinced  we 
have  the  need  to  deploy  a  3D  system(s)  to  any  of  the  typical  locations  we  deploy  to 
(which  normally  include  reach  back  or  local  "Host  nation"  supply  capabilities)...  I  will 
however  remain  optimistic  and  keep  an  open  mind. 

Round  3  Open-Response 

Per  #3,  personally  I  think  if  you  had  to  go  right  now  and  use  3D  printing,  I  would  consider 

starting  with  a  production  grade  polymer  printer  and  look  at  printing  fixtures,  tooling, 
and  models.  I  do  think  there  is  a  definite  role  for  metal  printers,  but  there  is  a  very  near- 
term  opportunity  for  polymer  systems. 

Per  #2,  reliability  is  right  up  with  usability.  Otherwise  if  the  3D  printer  breaks  down  enough 
times,  it  will  get  tucked  away  in  the  corner  and  never  get  used. 

My  own  bias  is  that  powder-based  methods  are  inherently  expensive  and  subject  to  internal 

flaws  that  are  not  visible  as-produced.  I  see  merit  in  free-form  welding  integrated  with 
local  machining.  (Full  disclosure,  I  am  working  to  try  and  develop  this  approach). 
Powders  are  messy,  difficult  to  handle,  require  cleaning/classification  to  be  recycled, 
and  often  require  post  processing  (such  as  HIPping). 

Polymers  may  be  useful  for  drill  guides,  fixturing,  brackets,  etc.  This  should  not  be  overlooked. 

In  general,  the  need  for  simplicity  and  robustness  are  key. 

7)  With  limited  knowledge  combined  with  no  experience  with  3D  printing  does  not  lead  me  to  a 
solid  conclusion  at  this  point... 

Some  of  these  questions  are  two  part  questions  and  can't  be  answered  with  one  response.  I  am 
still  of  the  belief  that  there  is  no  valid  requirement  for  a  3D  printer  in  the  deployed 
environment.  But  after  reading  these  questions  and  not  being  able  to  relate  to  the  topic 
I  too  feel  I  am  not  the  right  person  to  be  asking.  3D  printing  could  be  feasible  but  from 
my  experience  it  is  not. 

I  believe  the  group  consensus  is  on  target  for  the  most  part.  The  areas  where  I  have  some 
difference  in  opinion  are  in  regards  to  the  importance  of  binder  jetting  abilities  and 
training  requirements.  They  both  were  identified  as  secondary  points  and  I  am  able  to 
see  value  of  the  group  results  above  my  personal  thoughts.  All  and  all,  it  looks  to  me  to 
be  very  promising. 

Perhaps  a  pilot  study  should  start  with  a  survey  of  common  problems  or  issues  that  use  of  the 
UTC  confronts  in  everyday  use,  and  then  compose  creative  ways  in  which  a  3DPrinter 
could  solve  or  mitigate  those  problems.  This  would  help  define  the  requirements  of  the 
printer  implemented  in  the  UTC.  Then  application  of  the  printer  to  address  some  of 
those  scenarios  could  be  used  as  test  cases  as  part  of  the  evaluation.  In  other  words, 
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success  of  3DPri nti ng  as  a  part  of  a  UTC  might  depend  critically  on  careful  deliberation 
about  the  typical  uses  for  a  printer  in  the  field,  and  then  specifying  the  right  printer  and 
procedures  to  meet  those  requirements. 

Nice  job  on  the  study! 

Again,  I  don't  think  this  would  be  applicable  for  Fire  Department  use,  but  possibly  in  other  CE 
areas.  The  parts  created  or  printed  would  be  very  specific  and  beneficial,  but  the 
benefits  and  printing  would  have  to  outweigh  the  cost  of  shipping,  storage, 
maintenance,  training,  programming,  fuel,  electricity,  and  most  importantly,  the  cost, 
transportation,  and  storage  of  the  bulky  raw  materials  that  are  used  in  the  printing 
process.  With  the  drawdown  of  the  conflicts  in  this  area  and  the  availability  of  building 
materials  everywhere,  I  don't  see  this  as  practical.  In  my  16  %  years  in  the  Air  Force  and 
through  5  deployments,  I  have  never  heard  any  CE  AFSC  say  "I  wish  I  had  a  3D  printer." 

Concerning  pilot  or  test  study,  it  is  vital  that  collaboration  occur  amongst  the  CE  AFSCs  to 
identify/assess  the  total  needs  and  mobilization  requirements  for  the  platform 

In  the  mindset  of  efficiencies,  I  can't  imagine  that  CE  operations  or  emergencies  services 

individually  would  have  production  requirements  that  would  warrant  that  either  would 
require  an  independent  system. 

One  system  should  support  all  CE  at  one  deployed  location 

Round  4  Open-Response 

This  is  an  interesting  concept  and  I  look  forward  to  the  possibilities  that  the  study/ies  bring  to 
our  warfighters. 

I  don't  think  this  would  be  applicable  for  Fire  Department  use,  but  possibly  in  other  CE  areas. 

The  parts  created  or  printed  would  be  very  specific  and  beneficial,  but  the  benefits  and 
printing  would  have  to  outweigh  the  cost  of  shipping,  storage,  maintenance,  training, 
programming,  fuel,  electricity,  and  most  importantly,  the  cost,  transportation,  and 
storage  of  the  bulky  raw  materials  that  are  used  in  the  printing  process.  With  the 
drawdown  of  the  conflicts  in  this  area  and  the  availability  of  building  materials 
everywhere,  I  don't  see  this  as  practical.  In  my  16  ’/i  years  in  the  Air  Force  and  through  5 
deployments,  I  have  never  heard  any  CE  AFSC  say  "I  wish  I  had  a  3D  printer." 

With  that  said,  I  am  only  referring  to  this  not  working  in  a  UTC.  A  3D  printer  could  have  its  uses 
at  a  more  permanent/enduring  type  base  like  Al  Udeid  that  has  the  money, 
infrastructure,  storage,  software  updates,  and  raw  material  supply  train  to  sustain  an 
item  like  this. 

The  results  don't  necessarily  surprise  me  -  it  seems  most  participants  favor  the  technology  (with 
some  reservations). 

Thanks  and  can't  wait  to  see  the  full  dissertation  in  final  form.  Much  respect! 

Great  Job  in  exploring  new  technologies  that,  one  day,  could  help  CE  craftsman  perform  their 
mission  with  a  lighter  more  agile  footprint...  Thx  for  the  opportunity  to  provide 
feedback! 

"Usability"  covers  a  different  scope  depending  on  each  person's  perspective.  Narrowly  defined, 

I  would  add,  repeatability  to  the  additional  list. 

More  generally,  I  think  the  message  is  that  metals  3D  printers  are  more  suited  than  polymer 

printers  at  this  time  (i.e.  #3).  I  think  the  middle  rating  may  be  caused  by  debate  over  the 
best  metal  printing  rather  than  metals  vs  polymer.  Thus  I  agree  with  the  result. 
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